128

Is it possible to loop over tuples in bash?

As an example, it would be great if the following worked:

for (i,j) in ((c,3), (e,5)); do echo "$i and $j"; done

Is there a workaround that somehow lets me loop over tuples?

Frank
  • 64,140
  • 93
  • 237
  • 324
  • 6
    Coming from python background this is a very useful question indeed! – John Jiang Jan 25 '14 at 06:25
  • 5
    looking at this four years later I wonder if there is still no better way of doing this. omg. – Giszmo Jun 22 '16 at 23:54
  • Almost 8 years later I also wondered if there is still no better way of doing this. But this 2018 answer looks pretty good to me: https://stackoverflow.com/a/52228219/463994 – MountainX Dec 09 '19 at 00:06

12 Answers12

107
$ for i in c,3 e,5; do IFS=","; set -- $i; echo $1 and $2; done
c and 3
e and 5

About this use of set (from man builtins):

Any arguments remaining after option processing are treated as values for the positional parameters and are assigned, in order, to $1, $2, ... $n

The IFS="," sets the field separator so every $i gets segmented into $1 and $2 correctly.

Via this blog.

Edit: more correct version, as suggested by @SLACEDIAMOND:

$ OLDIFS=$IFS; IFS=','; for i in c,3 e,5; do set -- $i; echo $1 and $2; done; IFS=$OLDIFS
c and 3
e and 5
heemayl
  • 39,294
  • 7
  • 70
  • 76
Eduardo Ivanec
  • 11,668
  • 2
  • 39
  • 42
  • 8
    Nice -- just want to point out `IFS` should be saved and reset to its original value if this is run on the command line. Also, the new `IFS` can be set once, before the loop runs, rather than every iteration. – 0eggxactly Mar 15 '12 at 03:46
  • 1
    In case any of the $i starts with a hyphen, it's safer to `set -- $i` – glenn jackman Mar 15 '12 at 11:35
  • 2
    Instead of saving `IFS`, only set it for the `set` command: `for i in c,3 e,5; do IFS="," set -- $i; echo $1 and $2; done`. Please edit your answer: If all readers would choose only one of the listed solutions, there's no sense in having to read the full development history. Thanks for this cool trick! – cfi Oct 30 '15 at 08:43
  • 2
    If I declare `tuples="a,1 b,2 c,3"` and put `IFS=','` as in the edited version , and instead of `c,3 e,5` use `$tuples` it doesn't print well at all. But instead if I put `IFS=','` just after the `do` keyword in the for loop, it works well when using `$tuples` as well as litteral values. Just thought it was worth saying. – Simonlbc Jun 02 '16 at 15:51
  • @Simonlbc that's because the for loop uses `IFS` to split iterations. i.e. If you loop over an array like `arr=("c,3" "e,5")` and put `IFS` before the for loop, the value of `$i` will be just `c` and `e`, it will split away `3` and `5` so `set` won't parse correctly because `$i` won't have anything to parse. This mean that if the values to iterate are not inlined, the `IFS` should be put inside the loop, and the outside value should respect the intended separator for the variable to iterate upon. In the cases of `$tuples` it should be simply `IFS=` which is default and splits upon whitespace. – untore Mar 16 '17 at 18:44
70

Based on the answer given by @eduardo-ivanec without setting/resetting the IFS, one could simply do:

for i in "c 3" "e 5"
do
    set -- $i # convert the "tuple" into the param args $1 $2...
    echo $1 and $2
done

The output:

c and 3
e and 5
rogerdpack
  • 62,887
  • 36
  • 269
  • 388
MZHm
  • 2,388
  • 1
  • 18
  • 24
  • 3
    This approach seems to me a lot simpler than the accepted and most-upvoted approach. Is there any reason not to do it this way as opposed to what @Eduardo Ivanec suggested? – spurra Jan 29 '20 at 14:55
  • @spurra this answer is 6 years and ½ more recent, and based on it. Credit where it's due. – Diego Jul 13 '20 at 13:06
  • 1
    @Diego I am aware of that. It's explicitly written in the answer. I was asking if there is any reason not to use this approach over the accepted answer. – spurra Jul 13 '20 at 13:33
  • 4
    @spurra you'd want to use Eduardo's answer if the default separator (space, tab or newline) doesn't work for you for some reason (https://bash.cyberciti.biz/guide/$IFS) – Diego Jul 13 '20 at 13:45
  • What would you do if you already have a `$1`, `$2` being utilized within your function because you're passing parameters? i.e. how do you distinguish between the 1st & 2nd parameter passed on vs. the 1st & 2nd item of the tuples? – mfaani Sep 15 '22 at 01:56
  • IFS stands for internal field separator. Based on IFS `set` splits the string. https://tldp.org/LDP/abs/html/internalvariables.html#IFSREF – Bhavya Jain Sep 30 '22 at 14:27
51

This bash style guide illustrates how read can be used to split strings at a delimiter and assign them to individual variables. So using that technique you can parse the string and assign the variables with a one liner like the one in the loop below:

for i in c,3 e,5; do 
    IFS=',' read item1 item2 <<< "${i}"
    echo "${item1}" and "${item2}"
done
Grant Humphries
  • 2,696
  • 2
  • 23
  • 24
20

Use associative array (also known as dictionary / hashMap):

animals=(dog cat mouse)
declare -A sound=(
  [dog]=barks
  [cat]=purrs
  [mouse]=cheeps
)
declare -A size=(
  [dog]=big
  [cat]=medium
  [mouse]=small
)
for animal in "${animals[@]}"; do
  echo "$animal ${sound[$animal]} and it is ${size[$animal]}"
done
VasiliNovikov
  • 9,681
  • 4
  • 44
  • 62
  • 1
    FYI, this didn't work for me on Mac with `GNU bash, version 4.4.23(1)-release-(x86_64-apple-darwin17.5.0)`, which was installed via brew, so YMMV. – David Oct 18 '18 at 00:58
  • it did however work on `GNU bash, version 4.3.11(1)-release-(x86_64-pc-linux-gnu)` from Ubuntu 14.04 within docker container. – David Oct 18 '18 at 01:00
  • seems like older versions of bash or ones not supporting this feature work off index basing? where the key is a number rather than string. http://tldp.org/LDP/abs/html/declareref.html, and instead of `-A` we have `-a`. – David Oct 18 '18 at 01:04
  • David, seems so. I think you can try array indices to get "associativity" then. Like `declare -a indices=(1 2 3); declare -a sound=(barks purrs cheeps); declare -a size=(big medium small)` etc. Haven't tried it in terminal yet, but I think it should work. – VasiliNovikov Oct 18 '18 at 06:46
  • 1
    It appears associative array order of iterating over keys is random, so be careful...https://stackoverflow.com/questions/29161323/how-to-keep-associative-array-order – rogerdpack Oct 07 '21 at 15:23
  • 1
    @rogerdpack Thanks! I've removed the part of my answer that dealt with non-deterministic iteration order, and the current solution should always iterate exactly as you defined it. – VasiliNovikov Oct 07 '21 at 18:21
9
c=('a' 'c')
n=(3    4 )

for i in $(seq 0 $((${#c[*]}-1)))
do
    echo ${c[i]} ${n[i]}
done

Might sometimes be more handy.

To explain the ugly part, as noted in the comments:

seq 0 2 produces the sequence of numbers 0 1 2. $(cmd) is command substitution, so for this example the output of seq 0 2, which is the number sequence. But what is the upper bound, the $((${#c[*]}-1))?

$((somthing)) is arithmetic expansion, so $((3+4)) is 7 etc. Our Expression is ${#c[*]}-1, so something - 1. Pretty simple, if we know what ${#c[*]} is.

c is an array, c[*] is just the whole array, ${#c[*]} is the size of the array which is 2 in our case. Now we roll everything back: for i in $(seq 0 $((${#c[*]}-1))) is for i in $(seq 0 $((2-1))) is for i in $(seq 0 1) is for i in 0 1. Because the last element in the array has an index which is the length of the Array - 1.

Lèse majesté
  • 7,923
  • 2
  • 33
  • 44
user unknown
  • 35,537
  • 11
  • 75
  • 121
6

But what if the tuple is greater than the k/v that an associative array can hold? What if it's 3 or 4 elements? One could expand on this concept:

###---------------------------------------------------
### VARIABLES
###---------------------------------------------------
myVars=(
    'ya1,ya2,ya3,ya4'
    'ye1,ye2,ye3,ye4'
    'yo1,yo2,yo3,yo4'
    )


###---------------------------------------------------
### MAIN PROGRAM
###---------------------------------------------------
### Echo all elements in the array
###---
printf '\n\n%s\n' "Print all elements in the array..."
for dataRow in "${myVars[@]}"; do
    while IFS=',' read -r var1 var2 var3 var4; do
        printf '%s\n' "$var1 - $var2 - $var3 - $var4"
    done <<< "$dataRow"
done

Then the output would look something like:

$ ./assoc-array-tinkering.sh 

Print all elements in the array...
ya1 - ya2 - ya3 - ya4
ye1 - ye2 - ye3 - ye4
yo1 - yo2 - yo3 - yo4

And the number of elements are now without limit. Not looking for votes; just thinking out loud. REF1, REF2

todd_dsm
  • 918
  • 1
  • 14
  • 21
  • This one works for sh. Since newer version of macOS stops keeping update bash, this once is more suitable for scripts targeting macOS. – WeZZard Sep 18 '22 at 05:44
  • There are a few bourn-like shells worth exploring: 1) bash (more feature-rich than the original), 2) dash (posix-compliant version of the original). For EVERY new macOS install, I would recommend `brew install shellcheck bash dash bash-completion@2`. This will get the latest GNU bash - just like on Linux; so everything is the same everywhere :-) – todd_dsm Sep 18 '22 at 23:55
  • The bash completion is only necessary if you're still using bash as the system shell `echo $SHELL` find out. On a new macOS it should be ZSH; in that case, do yourself a favor and install Oh My ZSH - it's just superior. – todd_dsm Sep 18 '22 at 23:58
6
$ echo 'c,3;e,5;' | while IFS=',' read -d';' i j; do echo "$i and $j"; done
c and 3
e and 5
kev
  • 155,172
  • 47
  • 273
  • 272
4

Using GNU Parallel:

parallel echo {1} and {2} ::: c e :::+ 3 5

Or:

parallel -N2 echo {1} and {2} ::: c 3 e 5

Or:

parallel --colsep , echo {1} and {2} ::: c,3 e,5
Ole Tange
  • 31,768
  • 5
  • 86
  • 104
2
do echo $key $value
done < file_discriptor

for example:

$ while read key value; do echo $key $value ;done <<EOF
> c 3
> e 5
> EOF
c 3
e 5

$ echo -e 'c 3\ne 5' > file

$ while read key value; do echo $key $value ;done <file
c 3
e 5

$ echo -e 'c,3\ne,5' > file

$ while IFS=, read key value; do echo $key $value ;done <file
c 3
e 5
prodriguez903
  • 161
  • 1
  • 4
2

Using printf in a process substitution:

while read -r k v; do
    echo "Key $k has value: $v"
done < <(printf '%s\n' 'key1 val1' 'key2 val2' 'key3 val3')

Key key1 has value: val1
Key key2 has value: val2
Key key3 has value: val3

Above requires bash. If bash is not being used then use simple pipeline:

printf '%s\n' 'key1 val1' 'key2 val2' 'key3 val3' |
while read -r k v; do echo "Key $k has value: $v"; done
anubhava
  • 761,203
  • 64
  • 569
  • 643
1

In cases where my tuple definitions are more complex, I prefer to have them in a heredoc:

while IFS=", " read -ra arr; do
  echo "${arr[0]} and ${arr[1]}"
done <<EOM
c, 3
e, 5
EOM

This combines looping over lines of a heredoc with splitting the lines at some desired separating character.

bluenote10
  • 23,414
  • 14
  • 122
  • 178
0

A bit more involved, but may be useful:

a='((c,3), (e,5))'
IFS='()'; for t in $a; do [ -n "$t" ] && { IFS=','; set -- $t; [ -n "$1" ] && echo i=$1 j=$2; }; done
Diego Torres Milano
  • 65,697
  • 9
  • 111
  • 134