How to filter an array of objects based on values in an inner array with jq?

Question

Given this input:

[
  {
    "Id": "cb94e7a42732b598ad18a8f27454a886c1aa8bbba6167646d8f064cd86191e2b",
    "Names": [
      "condescending_jones",
      "loving_hoover"
    ]
  },
  {
    "Id": "186db739b7509eb0114a09e14bcd16bf637019860d23c4fc20e98cbe068b55aa",
    "Names": [
      "foo_data"
    ]
  },
  {
    "Id": "a4b7e6f5752d8dcb906a5901f7ab82e403b9dff4eaaeebea767a04bac4aada19",
    "Names": [
      "jovial_wozniak"
    ]
  },
  {
    "Id": "76b71c496556912012c20dc3cbd37a54a1f05bffad3d5e92466900a003fbb623",
    "Names": [
      "bar_data"
    ]
  }
]

I'm trying to construct a filter with jq that returns all objects with Ids that do not contain "data" in the inner Names array, with the output being newline-separated. For the above data, the output I'd like is:

cb94e7a42732b598ad18a8f27454a886c1aa8bbba6167646d8f064cd86191e2b
a4b7e6f5752d8dcb906a5901f7ab82e403b9dff4eaaeebea767a04bac4aada19

I think I'm somewhat close with this:

(. - select(.Names[] contains("data"))) | .[] .Id

but the select filter is not correct and it doesn't compile (get error: syntax error, unexpected IDENT).

score 663 · Accepted Answer · edited May 23 '17 at 12:02

663

Very close! In your select expression, you have to use a pipe (|) before contains.

This filter produces the expected output.

. - map(select(.Names[] | contains ("data"))) | .[] .Id

The jq Cookbook has an example of the syntax.

Filter objects based on the contents of a key

E.g., I only want objects whose genre key contains "house".
$ json='[{"genre":"deep house"}, {"genre": "progressive house"}, {"genre": "dubstep"}]'
$ echo "$json" | jq -c '.[] | select(.genre | contains("house"))'
{"genre":"deep house"}
{"genre":"progressive house"}

Colin D asks how to preserve the JSON structure of the array, so that the final output is a single JSON array rather than a stream of JSON objects.

The simplest way is to wrap the whole expression in an array constructor:

$ echo "$json" | jq -c '[ .[] | select( .genre | contains("house")) ]'
[{"genre":"deep house"},{"genre":"progressive house"}]

You can also use the map function:

$ echo "$json" | jq -c 'map(select(.genre | contains("house")))'
[{"genre":"deep house"},{"genre":"progressive house"}]

map unpacks the input array, applies the filter to every element, and creates a new array. In other words, map(f) is equivalent to [.[]|f].

edited May 23 '17 at 12:02

Community

1
1

answered Nov 02 '14 at 16:51

Iain Samuel McLean Elder

19,791
12
64
80

2

Thanks, works great! I did actually see that example, I just failed at adapting it to my scenario :-) – Abe Voelker Nov 02 '14 at 16:56
1

Is there anyway to "preserve the json structure of the array"? I like the genre example but it outputs two "json lines". I couldn't figure out the map part necessarily – Colin D Aug 25 '16 at 12:32
4

@ColinD I wasn't really happy with the reduce solution, so I replaced it with an explanation of the map function. Does that help? – Iain Samuel McLean Elder Sep 07 '16 at 10:31
@IainElder - What happens when the part of the search term (in this case house) is a variable? So say using --args term se. So contains("hou$term") – SnazzyBootMan May 03 '17 at 13:10
@Chris The variable `$term` would be treated as a string, so you should use string concatenation: `contains("hou" + $term)` – Iain Samuel McLean Elder May 03 '17 at 15:51
@IainElder - Of course! String concatenation, didn't realise it was supported - thanks. – SnazzyBootMan May 03 '17 at 18:48
what if `contains` is not OK for me because the string I look for is prefix of a string I _do not_ look for? – törzsmókus Oct 03 '19 at 10:05

score 36 · Answer 2 · edited Mar 27 '21 at 10:38

36

Here is another solution which uses any/2

map(select(any(.Names[]; contains("data"))|not)|.Id)[]

with the sample data and the -r option it produces:

cb94e7a42732b598ad18a8f27454a886c1aa8bbba6167646d8f064cd86191e2b
a4b7e6f5752d8dcb906a5901f7ab82e403b9dff4eaaeebea767a04bac4aada19

edited Mar 27 '21 at 10:38

syntagma

23,346
16
78
134

answered Aug 31 '17 at 00:53

jq170727

13,159
3
46
56

Exactly what I was looking for - why does this work with a semi-colon `.Names[] ; contains()` and not with a pipe `.Names[] | contains()`? – Matt Mar 12 '18 at 18:33
5

Ah, it's the `any(generator; condition)` form. I found that without using `any()` I would end up with duplicates in my results if `select()` matched more than once on the same object. – Matt Mar 12 '18 at 18:40

score 2 · Answer 3 · answered May 09 '23 at 14:51

2

Following jq map select expression produces the intended outcome:

aws ecr describe-images \
  --registry-id <aws_account_id> \
  --repository-name <ecr_repository_name> \
  --region <aws_region> \
  --no-cli-pager \
  --filter tagStatus=TAGGED \
| jq '.imageDetails | map(select(.imageTags[] | contains ("version_tag")))'

answered May 09 '23 at 14:51

Leia Renée

51
5

5

A bit more detail would be great – Bitfinicon May 13 '23 at 06:58

How to filter an array of objects based on values in an inner array with jq?

3 Answers3

Filter objects based on the contents of a key

Linked

Related