JQ JSON Creation

Question

New to JSON creation, decided to try JQ from this question. However, when I tried implementing it, all I received was the help page for JQ and these types of errors: ./test.sh: line 43: --: command not found for the last three lines.

#!/bin/bash

echo Starting

sensorType="XXX"
sensorLocation="XXX"
sensorCommand="XXXXX"

# Hardware information. 
systemTime=$(date +%d-%m-%Y_%H:%M:%S)
kernalName=$(uname -s)
nodeName=$(uname -i)
kernalRelease=$(uname -r)
kernalVersion=$(uname -v)
machine=$(uname -m)
processor=$(uname -p)
hardwarePlatform=$(uname -i)
operatingSystem=$(uname -o)
timeup=$(uptime)

# Software information.
softVersion=$(XXX version)

JSON_STRING=$( jq -n \
                -- arg systemTime "$systemTime" \
                -- arg sensorType "$sensorType" \
                -- arg sensorLocation "$sensorLocation" \
                -- arg sensorCommand "$sensorCommand" \
                -- arg kernalName "$kernalName" \
                -- arg nodeName "$nodeName" \
                -- arg kernalRelease "$kernalRelease" \
                -- arg kernalVersion "$kernalVersion" \ 
                -- arg machine "$machine" \ 
                -- arg processor "$processor"
                -- arg hardwarePlatform "$hardwarePlatform" \
                -- arg operatingSystem "$operatingSystem" \
                -- arg timeup "$timeup" \
                -- arg softVersion "$softVersion" \
                '{systemTime: $systemTime, sensorType: $sensorType, sensorLocation: $sensorLocation, kernalName: $kernalName, nodeName: $nodeName, kernalRelease: $kernalRelease, machine: $machine, processor: $processor, hardwarePlatform: $hardwarePlatform, operatingSystem: $operatingSystem, timeup: $timeup, softVersion: $softVersion }' )

echo $JSON_STRING

Not sure if this is the most efficient way of using JQ or if there is a better way to implement it. But I would love to hear if there is a more efficient / easier way to accomplish this.

It seems to be a typo in your `jq` command. There shouldn't be no spaces between the `--` and the `arg` field — Inian, Jan 24 '19 at 16:46
You're also missing a backslash on the `processor` line, which is what leads to the "command not found" error; the `--` for `hardwarePlatform` is seen as the start of a new command, not an argument of the `jq` command. — chepner, Jan 24 '19 at 17:30

score 1 · Answer 1 · answered Jan 24 '19 at 17:37

Use an array to store the arguments before calling jq; it's easier to spread the arguments across multiple lines in an array assignment, as the backslashes aren't necessary.

jq_args=(
   --arg systemTime "$systemTime"
   --arg sensorType "$sensorType"
   --arg sensorLocation "$sensorLocation"
   --arg sensorCommand "$sensorCommand"
   --arg kernalName "$kernalName"
   --arg nodeName "$nodeName"
   --arg kernalRelease "$kernalRelease"
   --arg kernalVersion "$kernalVersion" 
   --arg machine "$machine"
   --arg processor "$processor"
   --arg hardwarePlatform "$hardwarePlatform"
   --arg operatingSystem "$operatingSystem"
   --arg timeup "$timeup"
   --arg softVersion "$softVersion"
)
JSON_STRING=$( jq -n "${jq_args[@]}" '{
   systemTime: $systemTime,
   sensorType: $sensorType,
   sensorLocation: $sensorLocation,
   kernalName: $kernalName,
   nodeName: $nodeName,
   kernalRelease: $kernalRelease,
   machine: $machine,
   processor: $processor,
   hardwarePlatform: $hardwarePlatform,
   operatingSystem: $operatingSystem,
   timeup: $timeup,
   softVersion: $softVersion 
 }' )

If you are using a version of bash that supports associative arrays, you can further simply the building of jq_args:

declare -A x
x=([systemTime]="$systemTime"
   [sensorType]="$sensorType"
   # etc
  )

declare -a jq_args
for k in "${!x[@]}"; do
  jq_args+=(--arg "$k" "${x[$k]}")
done

JSON_STRING=$( jq -n "${jq_args[@]}" ... )

Little tip to reduce the filter a bit, `{ $foo }` is the same as doing `{ foo: $foo }`. So you could use this on all those properties. — Jeff Mercado, Feb 11 '19 at 08:27
Funny you should mention that; I just remembered that feature yesterday in another answer :) — chepner, Feb 11 '19 at 12:59

peak · Accepted Answer · 2019-01-24T18:16:47.423

The first problem, as @Inian pointed out, is that there should be no space between "--" and "arg".

The second problem is that there should be no spaces after the backslash when it is used (as here) for line-continuation: for the backslash to serve as a line-continuation character, it must escape (i.e. immediately precede) the newline.

Otherwise, except possibly for some oddities such as $(XXX version), you should be good to go, which is not to say there aren't better ways to create the JSON object. See the next section for an illustration of an alternative approach.

Illustration of an alternative approach

The following approach can be used even if the keys and/or values contain control characters:

FMT="%s\0%s\0"
(
printf $FMT systemTime $(date +%d-%m-%Y_%H:%M:%S)
printf $FMT newline "$(echo a; echo b)"
) | jq -sR '[split("\u0000") | range(0;length;2) as $i | {(.[$i]): .[$i + 1]}] | add'

If it is known that no keys or values contain literal newline characters, the following variant, which has the main advantage of not requiring the "-s" option, could be used:

(
echo systemTime
date +%d-%m-%Y_%H:%M:%S
echo uname
uname -a
echo softVersion
echo XXX version
) | jq -nR '[inputs as $key | {($key): input}] | add'

Thank you! The problem @Inian pointed out and the backslash got me. — Hustlin, Jan 24 '19 at 17:10

score 0 · Answer 3 · answered Jan 24 '19 at 18:04

Not sure if this is the most efficient way of using JQ

Here are two alternative approaches. The first is based on the TSV format, and the second the CSV format.

Using the TSV format

The following assumes that neither literal tabs nor literal newlines appear within keys or values.

FMT="%s\t%s\n"
(
printf $FMT systemTime $(date +%d-%m-%Y_%H:%M:%S)
printf $FMT uname "$(uname -a)"
printf $FMT softVersion "XXX version"
) | jq -nR '[inputs | split("\t") | {(.[0]): .[1]}] | add'

Using a CSV-to-JSON utility

The approach illustrated here is probably mainly of interest if one starts with a CSV file of key-value pairs.

In the following, we'll use the excellent csv2json utility at https://github.com/fadado/CSV There are actually two executables provided there. Both convert each CSV row to a JSON array. We'll use extended symlinked to csv2json.

(
echo systemTime, $(date +%d-%m-%Y_%H:%M:%S)
echo uname, "$(uname -a)"
echo softVersion, XXX version
) | csv2json 
  | jq -n '[inputs as [$key, $value] | {($key): $value}] | add'

Jeff Mercado · Answer 4 · 2019-01-26T00:18:03.993

If you have all the values as environment variables, you could just utilize the env object to get the values. The syntax then becomes trivial.

systemTime=$(date +%d-%m-%Y_%H:%M:%S) \
kernalName=$(uname -s) \
nodeName=$(uname -i) \
kernalRelease=$(uname -r) \
kernalVersion=$(uname -v) \
machine=$(uname -m) \
processor=$(uname -p) \
hardwarePlatform=$(uname -i) \
operatingSystem=$(uname -o) \
timeup=$(uptime) \
jq -n 'env | {
    systemTime,
    sensorType,
    sensorLocation,
    kernalName,
    nodeName,
    kernalRelease,
    machine,
    processor,
    hardwarePlatform,
    operatingSystem,
    timeup,
    softVersion 
}'

JQ JSON Creation

4 Answers4

Illustration of an alternative approach

Using the TSV format

Using a CSV-to-JSON utility