Looping through variable with spaces

Question

This piece of code works as expected:

for var in a 'b c' d; 
do 
    echo $var; 
done

The bash script loops through 3 arguments printing

a
b c
d

However, if this string is read in via jq , and then looped over like so:

JSON_FILE=path/to/jsonfile.json
ARGUMENTS=$(jq -r '.arguments' "${JSON_FILE}")

for var in ${ARGUMENTS};
do 
    echo $var; 
done

The result is 4 arguments as follows:

a
'b
c'
d

Example json file for reference:

{
    "arguments" : "a 'b c' d"
}

What is the reason for this? I tried putting quotes around the variable like suggested in other SO answers but that caused everything to just be handled as 1 argument.

What can I do to get the behavior of the first case (3 arguments)?

Does this answer your question? [Variable substitution in a for-loop using {$var}](https://stackoverflow.com/questions/5447302/variable-substitution-in-a-for-loop-using-var) — Jeff Mercado, Aug 08 '20 at 00:40
This has nothing to do with jq, it's strictly a shell issue. — Jeff Mercado, Aug 08 '20 at 00:42
You might be able to use `xargs` to parse the quoted strings (see my answer to the superuser question ["Get bash to respect quotes when word splitting subshell output"](https://superuser.com/questions/1529226/get-bash-to-respect-quotes-when-word-splitting-subshell-output/1529316)) (provided the quoting/escaping in the string matches `xargs`' syntax). This may be tricky to implement, depending on what's actually in the `while` loop, but at worst you should be able to parse it into a bash array, then loop over that with `for var in "${arrayvar[@]}"`. — Gordon Davisson, Aug 08 '20 at 01:22
If you want to use jq, consider changing the format of your arguments to an array then output the array contents to a while loop. https://stackoverflow.com/questions/33950596/iterating-through-json-array-in-shell-script — Jeff Mercado, Aug 08 '20 at 01:37
The Q seems a bit muddled. Having a JSON string like "a 'b c' d" seems improbable, but if that's what you really have, you would either have to parse it in jq according to your requirements (what are they?), or pass it to some external program to parse. — peak, Aug 08 '20 at 02:10

KamilCuk · Answer 1 · 2022-09-07T12:13:22.487

3

What is the reason for this?

The word splitting expansion is run over unquoted results of other expansions. Because ${ARGUMENTS} expansion in for var in ${ARGUMENTS}; is unquoted, word splitting is performed. No, word splitting ignores quotes resulted from variable expansion - it only cares about whitespaces.

What can I do to get the behavior of the first case (3 arguments)?

The good way™ would be to write your own parser, to parse the quotes inside the strings and split the argument depending on the quotes.

I advise to use xargs, it (by default, usually a confusing behavior) parses quotes in the input strings:

$ arguments="a 'b c' d"
$ echo "${arguments}" | xargs -n1 echo 
a
b c
d
# convert to array
$ readarray -d '' arr < <(<<<"${arguments}" xargs printf "%s\0")

As presented in the other answer, you may use eval, but please do not, eval is evil and will run expansions over the input string.

edited Sep 07 '22 at 12:13

answered Aug 09 '20 at 11:03

KamilCuk

120,984
8
59
111

1

`-n1` is not needed for `xargs` – milahu Sep 07 '22 at 10:42
`-t` is not needed for `readarray` which is an alias for [mapfile](https://linuxcommand.org/lc3_man_pages/mapfileh.html) – milahu Sep 07 '22 at 11:31
Indeed, in this case `-t` is not needed. – KamilCuk Sep 07 '22 at 12:13
`xargs` needs `-r` to fix the case `arguments=""` where `arr` should be `arr=()` not `arr=("")` – milahu Sep 07 '22 at 18:07

score -1 · Answer 2 · answered Sep 07 '22 at 11:51

-1

Change IFS to a new line to make it work:

...
IFS='\n'; for var in $ARGUMENTS;
do 
    echo $var; 
done

answered Sep 07 '22 at 11:51

Ivan

6,188
1
16
23

Looping through variable with spaces

2 Answers2