Accessing a JSON object in Bash - associative array / list / another model

Question

I have a Bash script which gets data in JSON, I want to be able to convert the JSON into an accessible structure - array / list / or other model which would be easy to parse the nested data.

Example:

{
  "SALUTATION": "Hello world",
  "SOMETHING": "bla bla bla Mr. Freeman"
}

I want to get the value like the following: echo ${arr[SOMETHING]}

[ Different approach is optional as well. ]

Answer 1

If you want key and value, and based on How do i convert a json object to key=value format in JQ, you can do:

$ jq -r "to_entries|map(\"\(.key)=\(.value|tostring)\")|.[]" file
SALUTATION=Hello world
SOMETHING=bla bla bla Mr. Freeman

In a more general way, you can store the values into an array myarray[key] = value like this, just by providing jq to the while with the while ... do; ... done < <(command) syntax:

declare -A myarray
while IFS="=" read -r key value
do
    myarray[$key]="$value"
done < <(jq -r 'to_entries|map("(.key)=(.value)")|.[]' file)

And then you can loop through the values like this:

for key in "${!myarray[@]}"
do
    echo "$key = ${myarray[$key]}"
done

For this given input, it returns:

SALUTATION = Hello world
SOMETHING = bla bla bla Mr. Freeman

Answer 2

Although this question is answered, I wasn't able to fully satiate my requirements from the posted answer. Here is a little write up that'll help any bash-newcomers.

Foreknowledge

A basic associative array declaration

#!/bin/bash

declare -A associativeArray=([key1]=val1 [key2]=val2)

You can also use quotes (', ") around the declaration, its keys, and values.

#!/bin/bash

declare -A 'associativeArray=([key1]=val1 [key2]=val2)'

And you can delimit each [key]=value pair via space or newline.

#!/bin/bash

declare -A associativeArray([key1]=value1
  ['key2']=value2 [key3]='value3'
  ['key4']='value2'               ["key5"]="value3"


  ["key6"]='value4'
  ['key7']="value5"
)

Depending on your quote variation, you may need to escape your string.

Using Indirection to access both key and value in an associative array

example () {
  local -A associativeArray=([key1]=val1 [key2]=val2)

  # print associative array
  local key value
  for key in "${!associativeArray[@]}"; do
    value="${associativeArray["$key"]}"
    printf '%s = %s' "$key" "$value"
  done
}

Running the example function

$ example
key2 = val2
key1 = val1

Knowing the aforementioned tidbits allows you to derive the following snippets:

---

The following examples will all have the result as the example above

String evaluation

#!/usr/bin/env bash

example () {
  local arrayAsString='associativeArray=([key1]=val1 [key2]=val2)'
  local -A "$arrayAsString"

  # print associative array
}

Piping your JSON into JQ

#!/usr/bin/env bash
# Note: usage of single quotes instead of double quotes for the jq
#       filter. The former is preferred to avoid issues with shell 
#       substitution of quoted strings.
example () {
  # Given the following JSON
  local json='{ "key1": "val1", "key2": "val2" }'

  # filter using `map` && `reduce`
  local filter='to_entries | map("[\(.key)]=\(.value)") |
    reduce .[] as $item ("associativeArray=("; . + ($item|@sh) + " ") + ")"'

  # Declare and assign separately to avoid masking return values.
  local arrayAsString;
  # Note: no encompassing quotation (")
  arrayAsString=$(jq --raw-output "${filter}" <<< "$json")
  local -A "$arrayAsString"

  # print associative array
}

jq -n / --null-input option + --argfile && redirection

#!/usr/bin/env bash

example () {
  # /path/to/file.json contains the same json as the first two examples
  local filter filename='/path/to/file.json'

  # including bash variable name in reduction
  filter='to_entries | map("[\(.key | @sh)]=\(.value | @sh) ")
    | "associativeArray=(" + add + ")"'

  # using --argfile && --null-input
  local -A "$(jq --raw-output --null-input --argfile file "$filename" \
    "\$filename | ${filter}")"

  # or for a more traceable declaration (using shellcheck or other) this
  # variation moves the variable name outside of the string

  # map definition && reduce replacement
  filter='[to_entries[]|"["+(.key|@sh)+"]="+(.value|@sh)]|"("+join(" ")+")"'
  
  # input redirection && --join-output
  local -A associativeArray=$(jq --join-output "${filter}" < "${filename}")
  
  # print associative array
}

---

Reviewing previous answers

@Ján Lalinský

To load JSON object into a bash associative array efficiently (without using loops in bash), one can use tool 'jq', as follows.

# first, load the json text into a variable:
json='{"SALUTATION": "Hello world", "SOMETHING": "bla bla bla Mr. Freeman"}'

# then, prepare associative array, I use 'aa':
unset aa
declare -A aa

# use jq to produce text defining name:value pairs in the bash format
# using @sh to properly escape the values
aacontent=$(jq -r '. | to_entries | .[] | "[\"" + .key + "\"]=" + (.value | @sh)' <<< "$json")

# string containing whole definition of aa in bash
aadef="aa=($aacontent)"

# load the definition (because values may contain LF characters, aadef must be in double quotes)
eval "$aadef"

# now we can access the values like this: echo "${aa[SOMETHING]}"

Warning: this uses eval, which is dangerous if the json input is from unknown source (may contain malicious shell commands that eval may execute).

This could be reduced to the following

example () {
  local json='{ "key1": "val1", "key2": "val2" }'
  local -A associativeArray="($(jq -r '. | to_entries | .[] |
    "[\"" + .key + "\"]=" + (.value | @sh)' <<< "$json"))"

  # print associative array
}

@fedorqui

If you want key and value, and based on How do i convert a json object to key=value format in JQ, you can do:

$ jq -r "to_entries|map(\"\(.key)=\(.value|tostring)\")|.[]" file
SALUTATION=Hello world
SOMETHING=bla bla bla Mr. Freeman

In a more general way, you can store the values into an array myarray[key] = value like this, just by providing jq to the while with the while ... do; ... done < <(command) syntax:

declare -A myarray
while IFS="=" read -r key value
do
    myarray[$key]="$value"
done < <(jq -r "to_entries|map(\"\(.key)=\(.value)\")|.[]" file)

And then you can loop through the values like this:

for key in "${!myarray[@]}"
do
    echo "$key = ${myarray[$key]}"
done

For this given input, it returns:

SALUTATION = Hello world
SOMETHING = bla bla bla Mr. Freeman

The main difference between this solution and my own is looping through the array in bash or in jq.

Each solution is valid and depending on your use case, one may be more useful then the other.

Answer 3

_{Context: This answer was written to be responsive to a question title which no longer exists.}.

---

The OP's question actually describes objects, vs arrays.

To be sure that we help other people coming in who are actually looking for help with JSON arrays, though, it's worth covering them explicitly.

---

For the safe-ish case where strings can't contain newlines (and when bash 4.0 or newer is in use), this works:

str='["Hello world", "bla bla bla Mr. Freeman"]'
readarray -t array <<<"$(jq -r '.[]' <<<"$str")"

To support older versions of bash, and strings with newlines, we get a bit fancier, using a NUL-delimited stream to read from jq:

str='["Hello world", "bla bla bla Mr. Freeman", "this is\ntwo lines"]'
array=( )
while IFS= read -r -d '' line; do
  array+=( "$line" )
done < <(jq -j '.[] | (. + "\u0000")')

Answer 4

This is how can it be done recursively:

#!/bin/bash

SOURCE="$PWD"
SETTINGS_FILE="$SOURCE/settings.json"
SETTINGS_JSON=`cat "$SETTINGS_FILE"`

declare -A SETTINGS

function get_settings() {
    local PARAMS="$#"
    local JSON=`jq -r "to_entries|map(\"\(.key)=\(.value|tostring)\")|.[]" <<< "$1"`
    local KEYS=''

    if [ $# -gt 1 ]; then
        KEYS="$2"
    fi

    while read -r PAIR; do
        local KEY=''

        if [ -z "$PAIR" ]; then
            break
        fi

        IFS== read PAIR_KEY PAIR_VALUE <<< "$PAIR"

        if [ -z "$KEYS" ]; then
            KEY="$PAIR_KEY"
        else
            KEY="$KEYS:$PAIR_KEY"
        fi

        if jq -e . >/dev/null 2>&1 <<< "$PAIR_VALUE"; then
            get_settings "$PAIR_VALUE" "$KEY"
        else
            SETTINGS["$KEY"]="$PAIR_VALUE"
        fi
    done <<< "$JSON"
}

To call it:

get_settings "$SETTINGS_JSON"

The array will be accessed like so:

${SETTINGS[grandparent:parent:child]}

Answer 5

To load JSON object into a bash associative array efficiently (without using loops in bash), one can use tool 'jq', as follows.

# first, load the json text into a variable:
json='{"SALUTATION": "Hello world", "SOMETHING": "bla bla bla Mr. Freeman"}'

# then, prepare associative array, I use 'aa':
unset aa
declare -A aa

# use jq to produce text defining name:value pairs in the bash format
# using @sh to properly escape the values
aacontent=$(jq -r '. | to_entries | .[] | "[\"" + .key + "\"]=" + (.value | @sh)' <<< "$json")

# string containing whole definition of aa in bash
aadef="aa=($aacontent)"

# load the definition (because values may contain LF characters, aadef must be in double quotes)
eval "$aadef"

# now we can access the values like this: echo "${aa[SOMETHING]}"

Warning: this uses eval, which is dangerous if the json input is from unknown source (may contain malicious shell commands that eval may execute).

Answer 6

Building on @HelpNeeder's solution (nice one btw)

His solution wasn't really working with integers, so i made some additions. Extended amount of condition checks, so it's fair to say some performance is sacrificed.

This version works with integers and also floating point values.

SOURCE="$PWD"
SETTINGS_FILE="./test2.json"
SETTINGS_JSON=`cat "$SETTINGS_FILE"`
declare -A SETTINGS

get_settings() {
    local PARAMS="$#"
  
    local JSON=`jq -r "to_entries|map(\"\(.key)=\(.value|tostring)\")|.[]" <<< "$1"`
    local KEYS=''

    if [ $# -gt 1 ]; then
  
        KEYS="$2"
    fi

    while read -r PAIR; do
        local KEY=''

        if [ -z "$PAIR" ]; then
            break
        fi

        IFS== read PAIR_KEY PAIR_VALUE <<< "$PAIR"

        if [ -z "$KEYS" ]; then
            KEY="$PAIR_KEY"
        else
            KEY="$KEYS:$PAIR_KEY"
        fi
                
              
                res=$(jq -e . 2>/dev/null <<< "$PAIR_VALUE")
                
                exitCode=$?
                check=`echo "$PAIR_VALUE" | grep -E ^\-?[0-9]*\.?[0-9]+$`
          # if [ "${res}" ] && [ $exitCode -eq "0" ] && [[ ! "${PAIR_VALUE}" == ?(-)+([0-9]) ]]  ALTERNATIVE, works only for integer (not floating point)
          if [ "${res}" ] && [ $exitCode -eq "0" ] && [[ "$check" == '' ]]
            then
                get_settings "$PAIR_VALUE" "$KEY"
               else
            SETTINGS["$KEY"]="$PAIR_VALUE"
        fi
           
       

    done <<< "$JSON"
}
get_settings "$SETTINGS_JSON"

Answer 7

Solution: use jq( it's a lightweight and flexible command-line JSON processor.).

In bash I'd rather assign JSONs object to a variable and use jq in order to access and parse the right result from it. It's more convenient than parse this structure with arrays and it comes out of the box with multiple functionalities and features such as accessing nested and complex objects, select methods, builtin operators and functions, regex support ,comparisons etc...

example:

example='{"SALUTATION": "Hello world","SOMETHING": "bla bla bla Mr. Freeman"}'
echo $example | jq .SOMETHING

# output:
"bla bla bla Mr. Freeman"