Constructing a JSON object from a bash associative array

Question

I would like to convert an associative array in bash to a JSON hash/dict. I would prefer to use JQ to do this as it is already a dependency and I can rely on it to produce well formed json. Could someone demonstrate how to achieve this?

#!/bin/bash

declare -A dict=()

dict["foo"]=1
dict["bar"]=2
dict["baz"]=3

for i in "${!dict[@]}"
do
    echo "key  : $i"
    echo "value: ${dict[$i]}"
done

echo 'desired output using jq: { "foo": 1, "bar": 2, "baz": 3 }'

Answer 1

There are many possibilities, but given that you already have written a bash for loop, you might like to begin with this variation of your script:

#!/bin/bash
# Requires bash with associative arrays
declare -A dict

dict["foo"]=1
dict["bar"]=2
dict["baz"]=3

for i in "${!dict[@]}"
do
    echo "$i" 
    echo "${dict[$i]}"
done |
jq -n -R 'reduce inputs as $i ({}; . + { ($i): (input|(tonumber? // .)) })'

The result reflects the ordering of keys produced by the bash for loop:

{
  "bar": 2,
  "baz": 3,
  "foo": 1
}

In general, the approach based on feeding jq the key-value pairs, with one key on a line followed by the corresponding value on the next line, has much to recommend it. A generic solution following this general scheme, but using NUL as the "line-end" character, is given below.

Keys and Values as JSON Entities

To make the above more generic, it would be better to present the keys and values as JSON entities. In the present case, we could write:

for i in "${!dict[@]}"
do
    echo "\"$i\""
    echo "${dict[$i]}"
done | 
jq -n 'reduce inputs as $i ({}; . + { ($i): input })'

Other Variations

JSON keys must be JSON strings, so it may take some work to ensure that the desired mapping from bash keys to JSON keys is implemented. Similar remarks apply to the mapping from bash array values to JSON values. One way to handle arbitrary bash keys would be to let jq do the conversion:

printf "%s" "$i" | jq -Rs .

You could of course do the same thing with the bash array values, and let jq check whether the value can be converted to a number or to some other JSON type as desired (e.g. using fromjson? // .).

A Generic Solution

Here is a generic solution along the lines mentioned in the jq FAQ and advocated by @CharlesDuffy. It uses NUL as the delimiter when passing the bash keys and values to jq, and has the advantage of only requiring one call to jq. If desired, the filter fromjson? // . can be omitted or replaced by another one.

declare -A dict=( [$'foo\naha']=$'a\nb' [bar]=2 [baz]=$'{"x":0}' )

for key in "${!dict[@]}"; do
    printf '%s\0%s\0' "$key" "${dict[$key]}"
done |
jq -Rs '
  split("\u0000")
  | . as $a
  | reduce range(0; length/2) as $i 
      ({}; . + {($a[2*$i]): ($a[2*$i + 1]|fromjson? // .)})'

Output:

{
  "foo\naha": "a\nb",
  "bar": 2,
  "baz": {
    "x": 0
  }
}

Answer 2

This answer is from nico103 on freenode #jq:

#!/bin/bash

declare -A dict=()

dict["foo"]=1
dict["bar"]=2
dict["baz"]=3

assoc2json() {
    declare -n v=$1
    printf '%s\0' "${!v[@]}" "${v[@]}" |
    jq -Rs 'split("\u0000") | . as $v | (length / 2) as $n | reduce range($n) as $idx ({}; .[$v[$idx]]=$v[$idx+$n])'
}

assoc2json dict

Answer 3

You can initialize a variable to an empty object {} and add the key/values {($key):$value} for each iteration, re-injecting the result in the same variable :

#!/bin/bash

declare -A dict=()

dict["foo"]=1
dict["bar"]=2
dict["baz"]=3

data='{}'

for i in "${!dict[@]}"
do
    data=$(jq -n --arg data "$data" \
                 --arg key "$i"     \
                 --arg value "${dict[$i]}" \
                 '$data | fromjson + { ($key) : ($value | tonumber) }')
done

echo "$data"

Answer 4

bash 5.2 introduces the @k parameter transformation which, makes this much easier. Like:

$ declare -A dict=([foo]=1 [bar]=2 [baz]=3)
$ jq -n '[$ARGS.positional | _nwise(2) | {(.[0]): .[1]}] | add' --args "${dict[@]@k}"
{
  "foo": "1",
  "bar": "2",
  "baz": "3"
}

Answer 5

This has been posted, and credited to nico103 on IRC, which is to say, me.

The thing that scares me, naturally, is that these associative array keys and values need quoting. Here's a start that requires some additional work to dequote keys and values:

function assoc2json {
    typeset -n v=$1
    printf '%q\n' "${!v[@]}" "${v[@]}" |
        jq -Rcn '[inputs] |
                . as $v |
                (length / 2) as $n |
                reduce range($n) as $idx ({}; .[$v[$idx]]=$v[$idx+$n])'
}


$ assoc2json a
{"foo\\ bar":"1","b":"bar\\ baz\\\"\\{\\}\\[\\]","c":"$'a\\nb'","d":"1"}
$

So now all that's needed is a jq function that removes the quotes, which come in several flavors:

• if the string starts with a single-quote (ksh) then it ends with a single quote and those need to be removed
• if the string starts with a dollar sign and a single-quote and ends in a double-quote, then those need to be removed and internal backslash escapes need to be unescaped
• else leave as-is

I leave this last iterm as an exercise for the reader.

I should note that I'm using printf here as the iterator!