Getting empty string after parsing bash associative array to json using jq

Question

I have a bash associative array containing dynamic data like:

declare -A assoc_array=([cluster_name]="cpod1" [site_name]="ppod1" [alarm_name]="alarm1")

I have to create the JSON data accordingly.

{
"name": "cluster_name",
"value": $assoc_array[cluster_name],
"regex": False
}
{
"name": "site_name",
"value": $assoc_array[site_name],
"regex": False
}
{
"name": "alert_name",
"value": $assoc_array[alert_name],
"regex": False
}

I have used the following code for it:

for i in "${!assoc_array[@]}"; do
    echo $i
    alarmjsonarray=$(echo ${alarmjsonarray}| jq --arg name "$i" \
                                                --arg value "${alarm_param[$i]}" \
                                                --arg isRegex "False" \
                                                '. + [{("name"):$name,("value"):$value,("isRegex"):$isRegex}]')
done
echo "alarmjsonarray" $alarmjsonarray

I am getting empty string from it. Can you please help me in it?

Can you please provide a [mre] that's complete enough to copy-and-paste without changes? Your current code requires `cluster_name`, `site_name`, etc to have values already. — Charles Duffy, May 11 '21 at 15:57
Also, `echo ${alarmjsonarray}` is in general an antipattern; see [I just assigned a variable, but `echo $variable` prints something else!](https://stackoverflow.com/questions/29378566/i-just-assigned-a-variable-but-echo-variable-shows-something-else) — Charles Duffy, May 11 '21 at 15:58
(and in general, I'd suggest writing your code to call `jq` _only once_ instead of calling it in a loop; you can stream your data on input to a single copy of jq and let it turn that into an array). — Charles Duffy, May 11 '21 at 15:58
By default, jq transforms each value on stdin with the expression given. You can also bypass that and write an expression that uses `inputs` explicitly. — Charles Duffy, May 11 '21 at 16:01
What guarantees do you have about values in your strings? Do you need to worry about them containing newlines? What about NULs? Are there other sigils like tabs you can safely use? — Charles Duffy, May 11 '21 at 16:03
@CharlesDuffy I am checking already about newlines and NULs before putting it into the array. I have changed the question for it. Please let me know the correct code. I am new to jq. — Aadhi Verma, May 11 '21 at 16:05

score 4 · Accepted Answer · answered May 11 '21 at 16:14

4

Assuming the ASCII record separator character \x1e can't occur in your data (and given your assertions that newlines will never be present), one way to handle this would be:

for key in "${!assoc_array[@]}"; do
  printf '%s\x1e%s\n' "$key" "${assoc_array[$key]}"
done | jq -Rn '
[
  inputs |
  split("\u001e") | .[0] as $key | .[1] as $value |
  {"name": $key, "value": $value, "regex": false}
]'

...feel free to change \x1e to a different character (like a tab) that can never exist, as appropriate.

answered May 11 '21 at 16:14

Charles Duffy

280,126
43
390
441

Can we store the output of it into some variable, as I have take var alarmjsonarray? – Aadhi Verma May 11 '21 at 16:17
`result=$(for ...; done | jq ...)` will store the stdout of the entire pipeline in `result`. – Charles Duffy May 11 '21 at 16:19
I used the following code: alarmjsonarray=$(for key in "${!assoc_array[@]}"; do printf '%s\x1e%s\n' "$key" "${assoc_array[$key]}" done | jq -Rn ' [ inputs | split("\u001e") | .[0] as $key | .[1] as $value | {"name": $key, "value": $value, "regex": false} ]') echo "alarmjsonarray" $alarmjsonarray and I got the following result: alarmjsonarray [] – Aadhi Verma May 11 '21 at 16:48
I'd need a [mre] to be able to address any issues -- tested this before posting it, works for me.. – Charles Duffy May 11 '21 at 17:10
@AadhiVerma, ...see the exact code I tested and its output at https://gist.github.com/charles-dyfis-net/7621542a383893e94798c0c9cc970abb – Charles Duffy May 11 '21 at 17:14
I got the result now, why it is enclosed in [], when it was expected to be only like : { "name": "alarm_name", "value": "alarm1", "regex": false }, { "name": "cluster_name", "value": "cpod1", "regex": false }, { "name": "site_name", "value": "ppod1", "regex": false } – Aadhi Verma May 11 '21 at 17:24
@AadhiVerma, if you don't want it enclosed in `[]` just take that out; take out the `[` and the `]` at the beginning and end of the jq expression. You can also take out both the `-n` argument and the `inputs |`. – Charles Duffy May 11 '21 at 17:25
If you don't mind, can you update your github code for that? – Aadhi Verma May 11 '21 at 17:27
Done. (BTW, the reason I put the `[]`s in was I saw you using them in your original code). – Charles Duffy May 11 '21 at 17:31
Okay... So there was just doing hint ad trial to get the desired the output. Now the output you got, is not comma separated. Desired output is like : { "name": "alarm_name", "value": "alarm1", "regex": false }, { "name": "cluster_name", "value": "cpod1", "regex": false }, { "name": "site_name", "value": "ppod1", "regex": false } thanks in advance... You saved me today, its just last modification I need from you. – Aadhi Verma May 11 '21 at 17:50
It's not JSON anymore if you don't have the starting `[` and `]` but still have commas. I'm not willing to help someone create data that is not valid JSON. – Charles Duffy May 11 '21 at 17:55
So let it be with JSON only, is there any way to make "regex":False instead of false? – Aadhi Verma May 11 '21 at 20:07
`False` isn't a valid JSON literal. Is this really being parsed as a Python literal, maybe, instead of as JSON? – Charles Duffy May 11 '21 at 20:14
To be clear, you can definitely have `"regex": "False"`, but you can't have `"regex": False`. See the format spec at https://www.json.org/json-en.html – Charles Duffy May 11 '21 at 20:15
So this JSON will be passed to ansible-playbook and from there it will be passed to some API which uses python. – Aadhi Verma May 12 '21 at 04:46
If the API is _written in_ Python, but _expects to read_ JSON, then it should be `false`. Only if the API accepts Python literals instead of wanting JSON at all is `False` correct -- and if that's your scenario, I'd want to _write your code in Python_ and use `repr()` instead of any `json`-library thing to dump the results. – Charles Duffy May 12 '21 at 12:12
Thanks @charlesduffy this code is working fine with me, only issue with me they need the --arg format of JQ as I have specified in question. Otherwise the code you have written here for creating JSON is working absolutely fine with me. So can you edit your answer or add another answer with the expected format? – Aadhi Verma May 12 '21 at 13:27
`--arg` prevents you from reusing a single instance, since to pass in new values to the existing variables, you need a new instance of jq. In this context, it's actually a bad idea; it means your code _has_ to be inefficient. Why do they "need" that format so badly? – Charles Duffy May 12 '21 at 13:31
(To be clear, I don't see your question anywhere describing using `--arg` as a requirement, merely as the existing implementation; the whole point of asking a SO question is to get advice on how to improve your existing implementation to resolve an issue. If there's a requirement, explain the reasoning, and we can try to figure out how to satisfy it without the performance impact). – Charles Duffy May 12 '21 at 13:36
thanks was your efforts. I found the issue with my code. And your code was also looking good. I kept it with me for future use. I am marking your answer as correct answer. Thanks for being kind. – Aadhi Verma May 12 '21 at 16:44

score 1 · Answer 2 · edited May 14 '21 at 19:01

1

#!/bin/bash
declare -A assoc_array=([cluster_name]="cpod1" [site_name]="ppod1" [alarm_name]="alarm1")
for i in "${!assoc_array[@]}"; do
  alarmjsonarray=$(echo "${alarmjsonarray:-[]}" |
                   jq --arg name "$i" \
                      --arg value "${assoc_array[$i]}" \
                      --arg isRegex "False" \
                      '. + [{("name"):$name,("value"):$value,("isRegex"):$isRegex}]')
done
echo "alarmjsonarray" "$alarmjsonarray"

array was supposed to be initialized before using it. I thought in bash, scope is there out of block also. Its more simpler way to generate json from bash associative array.

edited May 14 '21 at 19:01

Charles Duffy

280,126
43
390
441

answered May 12 '21 at 16:43

Aadhi Verma

172
2
11

Do make sure you're quoting your expansions -- if you run your code through http://shellcheck.net/, it'll point out the places where this needs to be done. Otherwise, for example, a string `" * "` in your file can be changed to `" a.txt b.txt c.txt "` (substituting filenames in your current directory). – Charles Duffy May 12 '21 at 16:48
Thanks for your suggestion. I have updated my code accordingly. You had been very helpful n kind to me. – Aadhi Verma May 14 '21 at 18:13

Getting empty string after parsing bash associative array to json using jq

2 Answers2