Bash jq; build array using primary key

Question

I have a bash while do done loop in a loop that outputs using this
echo "${KEY}: {\"${ATTRIBUTE}\": ${VALUE} }"

Input part 1

"PKEY1": {"rank": 1 }
"PKEY2": {"rank": 2 }
"PKEY3": {"rank": 4 }
"PKEY4": {"rank": 3 }

Input part 2

"PKEY1": {"attr": "xyz" }
"PKEY2": {"attr": "foo" }
"PKEY3": {"attr": "bar" }
"PKEY4": {"attr": "abc" }

Input part 3,4,5,etc...

How can I get an output file or echo to look like this in short

{
"PKEY1": {"rank": 1, "attr": "xyz" },
"PKEY2": {"rank": 2, "attr": "foo" },
"PKEY3": {"rank": 4, "attr": "bar" },
"PKEY4": {"rank": 3, "attr": "abc" }
}

The output you have indicated you want is a JSON object, not a JSON array, so you might want to modify the TITLE of the question. — peak, Feb 06 '19 at 16:04

score 2 · Accepted Answer · answered Feb 06 '19 at 10:40

2

What you want is merge JSONs. If you are using 1.4+ you can use: jq -s '.[0] * .[1]' file1 file2

You can find more answers here: How to merge 2 json file using jq?

answered Feb 06 '19 at 10:40

Guilherme Diego

136
2

Small nitpick his input is not really valid json. – RedX Feb 06 '19 at 10:54
1

easy fix, just wrap it in { } and add , if not the last line. – mikeytown2 Feb 06 '19 at 11:09
Side note that trying to directly output this to another file `>` doesn't work. Once again easy work around but it's a small bump I encountered. – mikeytown2 Feb 14 '19 at 07:56

peak · Answer 2 · 2019-02-06T15:10:17.537

The problem as stated can be broken down into two parts:

1) parsing the text of key : value pairs into valid JSON;

2) constructing the output

Since the format of the key:value text is not clearly specified, I'm going to assume that each key:value pair occurs on a separate line, and that the key/value pair can be recovered using the def:

def parse: 
  capture("\"(?<k>[^\"]*)\" *: *(?<v>.*)") | [.k, (.v | fromjson)];

The second part of the problem is readily solved using the following generic def:

def add_by(s; f; g):
  reduce s as $x (null; .[$x|f] += ($x|g));

Putting the pieces together, we can write:

add_by(inputs | parse; .[0]; .[1])

Invocation

With the above snippets in merge.jq, we can write:

jq -nR -f merge.jq part1.txt part2.txt part3.txt ...

Variant

If your jq library already contains:

def aggregate_by(s; f; g):
  reduce s as $x  (null; .[$x|f] += [$x|g]);

then you could use the above def of parse as follows:

aggregate_by(inputs | parse; .[0]; .[1])
| map_values(add)

Bash jq; build array using primary key

2 Answers2

Invocation

Variant