Unable to append items from a while loop to an empty array in bash

Question

I have read some entries regarding this question like in here or here, but I haven't been able to make my code work. It should be very simple.

I need to append items from a while loop, after a small transformation, to an empty list. My code is as follows:

folder='/path/to/directories/'

ls  $folder | while read dir ; do
    if [[ $dir =~ ANF-* ]]; then
        names=()

        ls $folder/$dir/FASTQS  | while read file ; do
            name=$(echo $file | cut -d "_" -f 1-3 )
            echo $name
            names+=("$name")
        done
        echo ${names[*]}   #Also tried echo ${names[@]}
    fi
done

The first 'echo' works so it gets through the conditional and into the second loop.

I have also tried using 'declare -a' to create the empty array.

If I try to append $file to the list it does not work either.

I think the problem is the appending action because if I create an array that is not empty I got the array's items in the second 'echo'.

Thanks a lot in advance. RX

See also [BashFAQ/024](https://mywiki.wooledge.org/BashFAQ/024). — Benjamin W., Mar 14 '19 at 12:56
As a side note, instead of using `ls` (the output of which is [notoriously brittle](https://mywiki.wooledge.org/ParsingLs)), you should use globs: `for dir in "$folder"/*; do`, which solves both the setting-variable-in-a-subprocess problem *and* is robust with respect to filenames. — Benjamin W., Mar 14 '19 at 12:57
Furthermore, `ANF-*` as a regular expression means `ANF`, `ANF-`, `ANF--`, `ANF---` etc., which probably isn't what you meant. If you meant "starts with `ANF-`, you should have used `^ANF-.*` (`.*` for "any number of any character"), or just `^ANF-` because `=~` checks submatches; or `[[ $dir == ANF-* ]]` (shell patterns instead of regex). — Benjamin W., Mar 14 '19 at 13:00
And finally, I don't think you need a nested loop; you could do something like `for file in "$folder"/ANF-*/FASTQS/*; do`, and depending on what the filenames look like, you could probably use [parameter expansion](https://www.gnu.org/software/bash/manual/bash.html#Shell-Parameter-Expansion) to get the parts you want. — Benjamin W., Mar 14 '19 at 13:03
I will try, but I don't think that's the problem, I got into the loop and I get the files I am interested in modified, the problem is with appending the names to the list. — Rachael, Mar 14 '19 at 13:08
You're changing `names` in a subprocess (the inner `| while read` creates one), then you access `names` outside of that subprocess (`echo ${names[*]}`), but all the changes from inside the subprocess are gone. The simplest fix is to change from `ls [...] | while read file; do [...]; done` to `while read file; do [...]; done < <(ls [...])`, but notice that this still uses many anti-patterns such as using the output of `ls` programmatically. — Benjamin W., Mar 14 '19 at 13:32

vfalcao · Answer 1 · 2019-03-14T13:20:09.960

0

Try to wrap your echo with double quotes:

folder='/path/to/directories/'

ls  $folder | while read dir ; do
    if [[ $dir =~ ANF-* ]]; then
        names=()
        local iteration=1
        ls $folder/$dir/FASTQS  | while read file ; do
            name=$(echo $file | cut -d "_" -f 1-3 )
            echo $name
            names+=("$name")
            echo "iteration $iteration"
            iteration=$((iteration+1))
            declare -p names
        done
        echo "${names[@]}"
        # you can show the array like this:
        declare -p names
    fi
done

Your names array is being altered, but when the loop terminates, the alterations go away. Look at the answers @Benjamin W. kindly pointed.

edited Mar 14 '19 at 13:20

answered Mar 14 '19 at 12:46

vfalcao

332
1
3
12

Thanks a lot, but I have added your modifications but it is still not working.. The items of the of the loop are not being added to the list. The loop is working I get the list of files of the directories I am interested in, and I am able to modify the names inside the loop. – Rachael Mar 14 '19 at 13:00
2

Quoting has nothing to do with it; the problem is modifying a variable in a subshell. – Benjamin W. Mar 14 '19 at 13:04
@Rachael, your question was edited by a moderator (@Benjamin W.), and it points to a former answer... have you checked it out ? – vfalcao Mar 14 '19 at 13:04
Not a moderator, just a fellow user :) And the question wasn't edited, but marked as a duplicate. – Benjamin W. Mar 14 '19 at 13:05
The problem is modified without any problem, the problem is appending the item not modifying it. – Rachael Mar 14 '19 at 13:09

Unable to append items from a while loop to an empty array in bash

1 Answers1