Assigning an Array Parsed With jq to Bash Script Array

Question

I parsed a json file with jq like this :

# cat test.json | jq '.logs' | jq '.[]' | jq '._id' | jq -s

It returns an array like this : [34,235,436,546,.....]

Using bash script i described an array :

# declare -a msgIds = ...

This array uses () instead of [] so when I pass the array given above to this array it won't work.

([324,32,45..]) this causes problem. If i remove the jq -s, an array forms with only 1 member in it.

Is there a way to solve this issue?

Answer 1

We can solve this problem by two ways. They are:

Input string:

// test.json
{
    "keys": ["key1","key2","key3"]
}

Approach 1:

1) Use jq -r (output raw strings, not JSON texts) .

KEYS=$(jq -r '.keys' test.json)
echo $KEYS
# Output: [ "key1", "key2", "key3" ]

2) Use @sh (Converts input string to a series of space-separated strings). It removes square brackets[], comma(,) from the string.

KEYS=$(<test.json jq -r '.keys | @sh')
echo $KEYS
# Output: 'key1' 'key2' 'key3'

3) Using tr to remove single quotes from the string output. To delete specific characters use the -d option in tr.

KEYS=$((<test.json jq -r '.keys | @sh')| tr -d \') 
echo $KEYS
# Output: key1 key2 key3

4) We can convert the comma-separated string to the array by placing our string output in a round bracket(). It also called compound Assignment, where we declare the array with a bunch of values.

ARRAYNAME=(value1 value2  .... valueN)

#!/bin/bash
KEYS=($((<test.json jq -r '.keys | @sh') | tr -d \'\"))

echo "Array size: " ${#KEYS[@]}
echo "Array elements: "${KEYS[@]}

# Output: 
# Array size:  3
# Array elements: key1 key2 key3

Approach 2:

1) Use jq -r to get the string output, then use tr to delete characters like square brackets, double quotes and comma.

#!/bin/bash
KEYS=$(jq -r '.keys' test.json  | tr -d '[],"')
echo $KEYS

# Output: key1 key2 key3

2) Then we can convert the comma-separated string to the array by placing our string output in a round bracket().

#!/bin/bash
KEYS=($(jq -r '.keys' test.json  | tr -d '[]," '))

echo "Array size: " ${#KEYS[@]}
echo "Array elements: "${KEYS[@]}

# Output:
# Array size:  3
# Array elements: key1 key2 key3

Answer 2

To correctly parse values that have spaces, newlines (or any other arbitrary characters) just use jq's @sh filter and bash's declare -a. (No need for a while read loop or any other pre-processing)

// foo.json
{"data": ["A B", "C'D", ""]}

str=$(jq -r '.data | @sh' foo.json)
declare -a arr="($str)"   # must be quoted like this

$ declare -p arr
declare -a arr=([0]="A B" [1]="C'D" [2]="")

---

The reason that this works correctly is that @sh will produce a space-separated list of shell-quoted words:

$ echo "$str"
'A B' 'C'\''D' ''

and this is exactly the format that declare expects for an array definition.

Answer 3

Use jq -r to output a string "raw", without JSON formatting, and use the @sh formatter to format your results as a string for shell consumption. Per the jq docs:

@sh:

The input is escaped suitable for use in a command-line for a POSIX shell. If the input is an array, the output will be a series of space-separated strings.

So can do e.g.

msgids=($(<test.json jq -r '.logs[]._id | @sh'))

and get the result you want.

Answer 4

From the jq FAQ (https://github.com/stedolan/jq/wiki/FAQ):

: How can a stream of JSON texts produced by jq be converted into a bash array of corresponding values?

A: One option would be to use mapfile (aka readarray), for example:

mapfile -t array <<< $(jq -c '.[]' input.json)

An alternative that might be indicative of what to do in other shells is to use read -r within a while loop. The following bash script populates an array, x, with JSON texts. The key points are the use of the -c option, and the use of the bash idiom while read -r value; do ... done < <(jq .......):

#!/bin/bash
x=()
while read -r value
do
  x+=("$value")
done < <(jq -c '.[]' input.json)

Answer 5

++ To resolve this, we can use a very simple approach:

++ Since I am not aware of you input file, I am creating a file input.json with the following contents:

input.json:

{
    "keys": ["key1","key2","key3"]
}

++ Use jq to get the value from the above file input.json:

Command: cat input.json | jq -r '.keys | @sh'

Output: 'key1' 'key2' 'key3'

Explanation: | @sh removes [ and "

++ To remove ' ' as well we use tr

command: cat input.json | jq -r '.keys | @sh' | tr -d \'

Explanation: use tr delete -d to remove '

++ To store this in a bash array we use () with `` and print it:

command:

KEYS=(`cat input.json | jq -r '.keys | @sh' | tr -d \'`)

To print all the entries of the array: echo "${KEYS[*]}"