Bash, using 2 arrays in for loop and then 2 arrays inside the for loop

Question

I have a piece of code that will echo the file in a folders that is described in two variables. This is fine and works. However, i now want to list the files on another folder structure. However i cannot get it to list.

DIRS="foo bar live1 live2"
SRCDIR="/mnt/nas/stage/live/live/"

# iterate over all directories
for dirname in "${DIRS}"; do
  # go to source sync directory
  cd "${SRCDIR}/${dirname}"
  # get latest mux file name
  filename=`ls -t *.mux 2>/dev/null |head -n 1`
  echo $filename

  ###There is additional code to work on the files here that uses $dirname

done

I would then like to add another directory structure;

DIRS1="foo bar live1 live2"
SRCDIR1="/mnt/nas/stage/live/live/"
DIRS2="test1 test2"
SRCDIR2="/mnt/nas/stage/test/test/"

i have got to for dirname in ${DIRS1} ${DIRS2} This works as i get a No such file or directory error, but then I cannot work out to change my CD variable to SRCDIR2, i did think to put another for loop

for dirpath in ${SRCDIR1} ${SRCDIR2}
cd "${dirpath}/${dirname}"

However this gives a cartesian response. How can i sperate out the to only get DIR1 looking in SRCDIR1 and DIR2 looking in SRCDIR2?

Looking at other answer, i may need to use an array but cannot see how to put this into the code.

`for dirname in "${DIRS}"` doesn't split `${DIRS}`. Either make it an array and use `"${DIRS[@]}"` or use `for dirname in $DIRS` — Barmar, May 25 '23 at 17:25
This would be easier to do in a language with multi-dimensional arrays. — Barmar, May 25 '23 at 17:27
Please read [correct-bash-and-shell-script-variable-capitalization](https://stackoverflow.com/questions/673055/correct-bash-and-shell-script-variable-capitalization) and fix the issues that http://shellcheck.net will tell you about. — Ed Morton, May 25 '23 at 17:59

markp-fuso · Accepted Answer · 2023-05-25T18:33:43.657

If you know for a fact directory names do not contain whitespace you could use a pair of arrays, eg:

  DIRS[1]="foo bar live1 live2"
SRCDIR[1]="/mnt/nas/stage/live/live/"
  DIRS[2]="test1 test2"
SRCDIR[2]="/mnt/nas/stage/test/test/"

for ((id=1; id<=2; id++))
do
    for dir in ${DIRS[id]}                             # no double quotes so that each entry is processed separately
    do
        echo "${SRCDIR[id]}${dir}"
    done
done

This generates:

/mnt/nas/stage/live/live/foo
/mnt/nas/stage/live/live/bar
/mnt/nas/stage/live/live/live1
/mnt/nas/stage/live/live/live2
/mnt/nas/stage/test/test/test1
/mnt/nas/stage/test/test/test2

If directory names could contain whitespace this solution (unquoted ${DIRS[id]}) does not work, consider:

  DIRS[1]="foo bar live1 live2 'foo bar live{1,2}'"    # 5th entry contains a pair of spaces
SRCDIR[1]="/mnt/nas/stage/live/live/"
  DIRS[2]="test1 test2"
SRCDIR[2]="/mnt/nas/stage/test/test/"

for ((id=1; id<=2; id++))
do
    for dir in ${DIRS[id]}                             # split on whitespace
    do
        echo "${SRCDIR[id]}${dir}"
    done
done

This generates:

/mnt/nas/stage/live/live/foo
/mnt/nas/stage/live/live/bar
/mnt/nas/stage/live/live/live1
/mnt/nas/stage/live/live/live2
/mnt/nas/stage/live/live/'foo                          # 5th entry
/mnt/nas/stage/live/live/bar                           # broken into
/mnt/nas/stage/live/live/live{1,2}'                    # 3 parts
/mnt/nas/stage/test/test/test1
/mnt/nas/stage/test/test/test2

If directory names can contain white space then another approach would be to use hard-coded array names and namerefs (declare -n), eg:

  DIRS1=(foo bar live1 live2 'foo bar live{1,2}')      # 5th entry contains a pair of spaces
SRCDIR1="/mnt/nas/stage/live/live/"
  DIRS2=(test1 test2)
SRCDIR2="/mnt/nas/stage/test/test/"

for ((id=1; id<=2; id++))
do
    declare -n dirs="DIRS${id}"                        # nameref; 1st time dirs is a pointer to DIRS1
    declare -n srcdir="SRCDIR${id}"                    # nameref; 1st time srcdir is a pointer to SRCDIR1

    for dir in "${dirs[@]}"
    do
        echo "${srcdir}${dir}"
    done
done

NOTE: namerefs require bash 4.2+

This generates:

/mnt/nas/stage/live/live/foo
/mnt/nas/stage/live/live/bar
/mnt/nas/stage/live/live/live1
/mnt/nas/stage/live/live/live2
/mnt/nas/stage/live/live/foo bar live{1,2}             # 5th entry maintains whitespace
/mnt/nas/stage/test/test/test1
/mnt/nas/stage/test/test/test2

As mentioned in comments, the ideal solution would be a pair of multi-dimensional arrays (eg, DIRS[1,1]="foo"); unfortunately bash does not support multi-dimensional arrays.

Yes, it is possible to simulate a multi-dimensional array via concatenation of indices (eg, DIRS[1-1]="foo") but the necessary code gets a bit messy. I'll leave it to OP to research this idea if it's of interest ...

Hey Mark, That is brilliant. There are no whitespaces so your first answer worked. Your second answer for including whitespace looks like what i was toying with, however could you please explain a couple of the nuances to me. first why did you use brackets instead of quotes. second, what is the reason for declaring the array. As always, astounded and grateful for the generosity of 'people of the world' on this forum. — Red_badger, May 27 '23 at 15:02
@Red_badger not sure I understand either question; if you're referring to the square brackets (`[`, `]`), that's the syntax for encompassing an array index (`array_name[index]`); the only thing I explicitly declare are the namrefs (`declare -n`) — markp-fuso, May 27 '23 at 17:37

score 0 · Answer 2 · answered May 25 '23 at 21:18

Given the limitations of Bash arrays and associative arrays, and the many pitfalls associated with using variables indirectly, I'd consider using something like this Shellcheck-clean code:

#! /bin/bash -p

#               SRCDIR                      DIRS
#               --------------------------  -----------------------
srcdir_dirs=(   /mnt/nas/stage/live/live    foo bar live1 live2 ''
                /mnt/nas/stage/test/test    'test 1' 'test 2' ''    )

srcdir='' dirname=''
for d in "${srcdir_dirs[@]}"; do
    if [[ -z $d ]]; then
        srcdir=''
    elif [[ -z $srcdir ]]; then
        srcdir=$d
    else
        dirname=$d
        printf '%s/%s\n' "$srcdir" "$dirname"
        # ...
    fi
done

This works with all but ancient versions of Bash (tested with Bash 3 and Bash 5).
I've replaced test1 and test2 with test 1 and test 2 to demonstrate that names containing whitespace do not cause problems. Even newlines in names would be OK.
The empty string ('') entries in srcdir_dirs delimit groups of entries that define a source directory and multiple subdirectories. This is safe because the empty string is not a valid file or directory name or path.
The srcdir_dirs array can easily be extended to handle three or more source directories.
See the accepted, and excellent, answer to Why is printf better than echo? for an explanation of why I used printf instead of echo.

thanks for talking the time to answer PJH. i may experiment with this later. — Red_badger, May 27 '23 at 15:08

Bash, using 2 arrays in for loop and then 2 arrays inside the for loop

2 Answers2