Bash variable scope issue

Question

I am struggling to understand what the cause of the following bug is and how I can fix it.

I have this code:

set_filters() {
  json=$1
  filters='"Name=instance-state-name,Values=running,stopped"'
  echo $json | jq -r '. | keys[]' | \
    while read tag ; do
      value=$(echo "$json" | jq -r ".[\"$tag\"]")
      filters="$filters \"Name=tag:${tag},Values=${value}\""
    done
  echo $filters
}

set_filters '{"Name": "*FOO*", "Cost Center": "XX111"}'

The output I am expecting:

"Name=instance-state-name,Values=running,stopped" "Name=tag:Cost Center,Values=XX111" "Name=tag:Name,Values=*FOO*"

The output I am getting:

"Name=instance-state-name,Values=running,stopped"

If I insert echo statements to assist with debugging:

set_filters() {
  json=$1
  filters='"Name=instance-state-name,Values=running,stopped"'
  echo $json | jq -r '. | keys[]' | \
    while read tag ; do
      value=$(echo "$json" | jq -r ".[\"$tag\"]")
      filters="$filters \"Name=tag:${tag},Values=${value}\""
      echo "FILTERS INSIDE LOOP: $filters"
    done
  echo "FILTERS OUTSIDE LOOP: $filters"
}

The output I then get is:

FILTERS INSIDE LOOP: "Name=instance-state-name,Values=running,stopped" "Name=tag:Cost Center,Values=XX111"
FILTERS INSIDE LOOP: "Name=instance-state-name,Values=running,stopped" "Name=tag:Cost Center,Values=XX111" "Name=tag:Name,Values=*FOO*"
FILTERS OUTSIDE LOOP: "Name=instance-state-name,Values=running,stopped"

I can't explain the behaviour. In a language other than Bash I would assume a variable scope issue for the variable $filters, but I thought the scope would basically be global.

I am using JQ version 1.3 and Bash version 4.1.2 on Red Hat Enterprise Linux 6.8.

The jq tag was added by another editor. The jq code here is doing what it is expected to do so this isn't really a jq question. — Alex Harvey, Nov 23 '17 at 01:57
Agreed - I'd say you're clear to remove that tag when you make another edit. Of course it would help clarify the question if you could change the script to use something other than `jq`, while still demonstrating this strange behavior. — David Z, Nov 23 '17 at 02:15
Actually, I think this is almost *more* of a `jq` question, since you can probably dispense with all of the wrappers around `jq` and use a single `jq` command instead: consider the output of `echo "$1" | jq '.keys as $k | "Name=tag:\($k),Values=\(.[$k])"`. — chepner, Nov 23 '17 at 17:56

score 2 · Accepted Answer · answered Nov 23 '17 at 02:14

Bash executes loops in a subshell if they are part of a pipeline. See for example BashFAQ/024 and "Bash Script: While-Loop Subshell Dilemma".

A possible workaround is to use process substitution:

while read tag; do
  ...
done < <(jq -r '. | keys[]' <<< "$1")

Bash variable scope issue

1 Answers1