4

I have a aws cli bash loop generating multiple json lists, containing dictionaries. I can't get jq to merge the lists into one large list with all the dictionaries.

e.g.

[  { "key1": "value1", "key2": "value2" },
   { "key1": "value3", "key2": "value4" }
]
[  { "key1": "value5", "key2": "value6" }
]
[  { "key1": "value7" }
]

this gets piped to jq and i would like it to be merged to

[ { "key1": "value1", "key2": "value2" },
  { "key1": "value3", "key2": "value4" },
  { "key1": "value5", "key2": "value6" },
  { "key1": "value7" }
]
Pieter
  • 1,916
  • 17
  • 17
  • 2
    The snippet is not valid JSON. Properly quote the values in the input shown – Inian Aug 13 '20 at 10:42
  • No, it isn't - https://jsonlint.com/, try pasting it here – Inian Aug 13 '20 at 11:17
  • 3
    `jq -s add filename` works fine. – oguz ismail Aug 13 '20 at 11:20
  • I checked it there, the 2nd bit (result) is valid, the first is just 3 json lists, that is the issue. The bash loop runs aws cli and gets a new json list back everytime, i am trying to merge them into one using jq – Pieter Aug 13 '20 at 11:22

2 Answers2

5

Assuming you have jq 1.5, the solution posted here:

JQ How to merge multiple objects into one

provides the basic technique of using jq -n with the inputs operator.

cat FILE | jq -n '[inputs|.[]]'

does the trick. [inputs] glues the multiple results into a single list and the '.[]' strips the extra level of list that was added.

J Quinn
  • 319
  • 2
  • 8
-2

i hacked a fix using awk, there must me a way to do it with jq.

| awk 'NR==1{print}NR>1{sub(/^]/,"");sub(/^\[/,",");print}END{print "]"}' \

What it does,

  1. skip first line "[" (leave it in place)
  2. remove all "^]" at start of lines
  3. replace "[" at start of line with "," new list starting
  4. at the end close it all with "]"
Pieter
  • 1,916
  • 17
  • 17