Why is my list being stripped of commas?

Question

I'm trying to make a menu function, and have the following script segment, which is meant to split a comma separated list of values into an array:

IFS=,
read -r -a optionList <<< $3

However, the result is always a single entry with no commas

echo ${#optionList[@]}
echo ${optionList[0]}

Gives an output that looks like this:

1
Option 1 Option 2 Option 3

Yet the following is as expected:

echo $3

yields an output of:

Option 1,Option 2,Option 3

What am I missing here?

Note that the canonical way would be: `IFS=, read -r -d '' -a optionList < <(printf '%s,\0' "$3")`. — gniourf_gniourf, Mar 09 '16 at 07:35

score 4 · Answer 1 · edited Mar 09 '16 at 12:45

4

Quote variable $3.

When you set IFS session-wise, and do not quote any variable, that variable would suffer from word splitting according to the value(s) of IFS (, in this case). This is being done by shell before read gets the variable values hence read gets Option 1 Option 2 Option 3 instead of desired Option 1,Option 2,Option 3 to operate on:

$ IFS=,; echo $bar
Option 1 Option 2 Option 3

$ IFS=,; echo "$bar"
Option 1,Option 2,Option 3

With your problem:

$ echo $bar
Option 1 Option 2 Option 3


#### Without quoting

$ IFS=, read -ra foo <<< $bar

$ echo "${#foo[@]}"
1

$ echo "${foo[0]}"
Option 1 Option 2 Option 3


#### With Quoting

$ IFS=, read -ra foo <<< "$bar"

$ echo "${#foo[@]}"
3

$ echo "${foo[0]}"
Option 1

$ echo "${foo[1]}"
Option 2

$ echo "${foo[2]}"
Option 3

edited Mar 09 '16 at 12:45

chepner

497,756
71
530
681

answered Mar 09 '16 at 04:48

heemayl

39,294
7
70
76

Thanks! That worked! If you don't mind, could you tell me what exactly is happening when it isn't quoted? Is this something to do with all the different kinds of expansion that get done everywhere, that I still don't fully understand? – BryKKan Mar 09 '16 at 04:58
@Bryan Check my edits – heemayl Mar 09 '16 at 05:11
`IFS=,; read ...` will also cause leakage of `IFS` in current shell environment and will affect other shell commands – anubhava Mar 09 '16 at 05:22
@anubhava Right..thats what i called `session-wise` in my answer.. – heemayl Mar 09 '16 at 05:23
@heemayl Thanks. Both you and anubhava have been very helpful. – BryKKan Mar 09 '16 at 05:33
This is the right answer, but it's only to work around a bug introduced in `bash` 4.0. Leaving `$3` unquote works in 3.x, and the bug was fixed in 4.3. – chepner Mar 09 '16 at 12:44
I removed the semicolon separating `IFS` from `read`, since it is legal and preferable to restrict the change in `IFS` to the `read` command. – chepner Mar 09 '16 at 12:46

score 2 · Answer 2 · answered Mar 09 '16 at 04:47

2

You can use IFS inline as:

str='Option 1,Option 2,Option 3'
IFS=, read -ra optionList <<< "$str"

This will avoid corrupting IFS for current shell.

Check the output:

declare -p optionList
declare -a optionList='([0]="Option 1" [1]="Option 2" [2]="Option 3")'

answered Mar 09 '16 at 04:47

anubhava

761,203
64
569
643

Thanks! I didn't realize you could assign a value in the same line like that! – BryKKan Mar 09 '16 at 04:56
2

Yes `IFS` should not be set globally in shell to avoid impacting other command's behavior. – anubhava Mar 09 '16 at 04:58
I did a little research and ended up using && as I'd rather get no array at all than one using the wrong delimiter – BryKKan Mar 09 '16 at 05:00
That will work but it's not right as it will still change `IFS` in your current shell. You should first reset `IFS` using `unset IFS` then use: `IFS=, read -ra optionList <<< "$3"` – anubhava Mar 09 '16 at 05:03

score 2 · Accepted Answer · answered Mar 09 '16 at 05:05

There are two basic problems here: setting IFS affects a lot of things in how the shell parses strings, including some you're not expecting; and leaving $3 unquoted allows it to be one of the things that gets messed up.

What's happening is that when you refer to $3 without double-quotes around it, the shell splits it into "words" using $IFS as separator(s). In your example, this yields three words: "Option 1", "Option 2", and "Option 3". In the context of <<<, it then splices them back together using a space as a separator (which is weird, but that's what it does), yielding "Option 1 Option 2 Option 3". You can see this effect separately:

$ foo="Option 1,Option 2,Option 3"
$ IFS=,
$ cat <<< $foo
Option 1 Option 2 Option 3

So then the read command gets "Option 1 Option 2 Option 3" as input, splits it based on $IFS (","), finds that there are no word separators, so it puts it all in element #0 of optionList.

You can fix this by double-quoting $3 to prevent word-splitting, but you're going to have other problems because IFS is also going to affect lots of other things throughout your script. So you should also follow anubhava's recommendation to make the IFS assignment a prefix on the read command, so that it only affects that one command rather than being a global change to how the shell parses strings.

So, with both changes, my recommendation is to use:

IFS=, read -ra optionList <<< "$3"

... just like anubhava's recommendation.

I upvoted the other answers as they were helpful, but I am accepting yours as the most complete. Especially since you mentioned the searchable term "prefix" that helped me learn more about command prefixes generally. — BryKKan, Mar 09 '16 at 05:35

Why is my list being stripped of commas?

3 Answers3