Tokenizing a string in bash without using redirect

Question

Well in my script I use a simple line to tokenize:

IFS=';' read -ra ALL_ADDR <<< "${SERVER}"

However the moment I run this script in a busybox (alpine) linux docker container, it stumbles around above line. line 43: syntax error: unexpected redirection

I understand this is because the script is not run by Bash but rather Ash, which does not have the <<< redirect parameter. How can I transform the above line into code that works similar around all popular shell scripts. (Mainly Bash, Ash and Dash)?

for completeness the variable is like:

SERVER="/api/v1:127.0.0.1:1337;/api/v2:127.0.0.1:1338"
IFS=';' read -ra ALL_ADDR <<< "${SERVER}"
for i in "${ALL_ADDR[@]}"; do
    IFS=':' read -ra ADDR <<< "${i}"
    PROXY=${ADDR[0]}
    IPADDR=${ADDR[1]}
    PORT=${ADDR[2]}
    LOC_STRING="${LOC_STRING}\tlocation ${PROXY} {\n\t\tproxy_pass http://${IPADDR}:${PORT}\n\t}\n"
done
LOC_STRING=${LOC_STRING//\//\\\/}

#"insert" above generated lines into a site config for nginx
sed -e "0,/^\s*location/{s/^\s*location/${LOC_STRING}\n&/}" 'portal' > "../sites-enabled/portal"

Does this answer your question? [Bash: How to tokenize a string variable?](https://stackoverflow.com/questions/5382712/bash-how-to-tokenize-a-string-variable) — x3l51, Jan 08 '20 at 14:01
None of the answers in that question are particularly safe, and some of them use arrays which `ash` does not support. — chepner, Jan 08 '20 at 14:08
You could use an here-doc redirection for the same effect, but the redirection problem isn't the only one : `ash`'s `read` doesn't supports the `-a` flag. You are likely going to have to use a `while read -r ADDR; do [...]; done < — Aaron, Jan 08 '20 at 14:26

KamilCuk · Answer 1 · 2020-01-08T17:30:01.093

You don't seem to really need to tokenize it. Just generate the resulting string.

Below I first substitute each : or ; to a newline. Then with xargs I read 3 variables at a time and pass to printf format. The sed 's@/@\\/@g' does the same as LOC_STRING=${LOC_STRING//\//\\\/}.

SERVER="/api/v1:127.0.0.1:1337;/api/v2:127.0.0.1:1338"
LOC_STRING=$(
    printf "%s\n" "$SERVER"  |
    tr ':;' '\n\n' |
    xargs -d$'\n' -n3 printf '\tlocation %s {\\n\t\tproxy_pass http://%s:%s\\n\t}\\n' |
    sed 's@/@\\/@g'
)

echo "$LOC_STRING"

outputs:

    location \/api\/v1 {\n      proxy_pass http:\/\/127.0.0.1:1337\n  }\n location \/api\/v2 {\n      proxy_pass http:\/\/127.0.0.1:1338\n  }\n

To "tokenize" it it is simplest to insert a newline in place of the separator, and read it as a newline separated stream. There will be problems if in the string has a newline by itself, but judging from IFS=';' read -ra ALL_ADDR <<< "${SERVER}" there are no newlines in SERVER or you are interested in first line only. So similar - substitute ; and : for a newline. Then just read 3 items at a time. Use << to redirect something in posix shell.

tmp=$(printf "%s\n" "$SERVER" | tr ';:' '\n\n')
while IFS= read -r PROXY &&
        IFS= read -r IPADDR &&
        IFS= read -r POST; do
    LOC_STRING="${LOC_STRING}\tlocation ${PROXY} {\n\t\tproxy_pass http://${IPADDR}:${PORT}\n\t}\n"
done <<EOF
$tmp
EOF
LOC_STRING=${LOC_STRING//\//\\\/}

Note: by convention upper case variables are used for variables that will be exported. For local variables in script use lower case variables.

score 0 · Answer 2 · answered Jan 08 '20 at 17:36

0

Since this is just asking about the herestring, I'll just mention in passing that you really shouldn't use arrays in a script intended to be portable. Ignoring that issue for the moment, you can always replace the herestring with a heredoc:

IFS=';' read -ra ALL_ADDR << EOF
${SERVER}
EOF

Note that you really shouldn't be yelling, and this would be better written:

IFS=';' read -ra all_addr << EOF
${server}
EOF

answered Jan 08 '20 at 17:36

William Pursell

204,365
48
270
300

`ash` doesn't support arrays, either, though. – chepner Jan 08 '20 at 17:53
@chepner Yes, hence the first sentence! Perhaps I should be more explicit. – William Pursell Jan 08 '20 at 19:40

tshiono · Answer 3 · 2020-01-08T23:58:41.333

I may not be directly answering your question, but how about a sed solution:

SERVER="/api/v1:127.0.0.1:1337;/api/v2:127.0.0.1:1338"
LOC_STRING=$(echo "$SERVER" | tr ';' '\n' | sed -e 's#\([^:]*\):\([^:]*\):\([^:]*\)#\tlocation \1 {\n\t\tproxy_pass http://\2:\3\n\t}#' -e 's#/#\\/#g')

Output of echo "$LOC_STRING":

    location \/api\/v1 {
        proxy_pass http:\/\/127.0.0.1:1337
    }
    location \/api\/v2 {
        proxy_pass http:\/\/127.0.0.1:1338
    }

Note that the word \n within double quotes are decoded as a newline character in ash while it is left as is in bash. A little tweaking may be needed depending on your requirement.

Tokenizing a string in bash without using redirect

3 Answers3