jq: selecting a subset of keys from an object

Question

Given an input json string of keys from an array, return an object with only the entries that had keys in the original object and in the input array.

I have a solution but I think that it isn't elegant ({($k):$input[$k]} feels especially clunky...) and that this is a chance for me to learn.

jq -n '{"1":"a","2":"b","3":"c"}'   \
    | jq --arg keys '["1","3","4"]' \
    '. as $input 
     | ( $keys | fromjson )
     | map( . as $k
          | $input
          | select(has($k))
          | {($k):$input[$k]}
          )
     | add'

Any ideas how to clean this up?

I feel like Extracting selected properties from a nested JSON object with jq is a good starting place but i cannot get it to work.

It wasn't clear from the documentation that I could use ( ) on the rhs of the == statement via the `select(.key == ("1","3","4"))` — Jon, Jun 15 '15 at 04:39

score 36 · Answer 1 · answered Sep 19 '17 at 06:09

36

solution with inside check:

jq 'with_entries(select([.key] | inside(["key1", "key2"])))'

answered Sep 19 '17 at 06:09

user5672998

1,044
3
14
18

7

FYI, this returns any entry that starts with `key1` or `key2`, so also `key1234` and `key22`. – Elte Hupkes Mar 28 '19 at 11:37

score 13 · Accepted Answer · answered Sep 19 '17 at 06:24

the inside operator works for most of time; however, I just found the inside operator has side effect, sometimes it selected keys not desired, suppose input is { "key1": val1, "key2": val2, "key12": val12 } and select by inside(["key12"]) it will select both "key1" and "key12"

use the in operator if need an exact match: like this will select .key2 and .key12 only

jq 'with_entries(select(.key | in({"key2":1, "key12":1})))'

because the in operator checks key from an object only (or index exists? from an array), here it has to be written in an object syntax, with desired keys as keys, but values do not matter; the use of in operator is not a perfect one for this purpose, I would like to see the Javascript ES6 includes API's reverse version to be implemented as jq builtin

https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Array/includes

jq 'with_entries(select(.key | included(["key2", "key12"])))'

to check an item .key is included? from an array

but can I have two answers on the same thread? just because there're two different approaches — user5672998, Sep 22 '17 at 01:12
You can, but usually you want to put caveats to an approach in the same answer that suggests it, so no one has to read the entire Q+A before deciding on a suitable approach. And fairly similar approaches can often be combined into one answer. — Nathan Tuggy, Sep 22 '17 at 01:19

Jeff Mercado · Answer 3 · 2018-08-21T20:48:30.797

12

You can use this filter:

with_entries(
    select(
        .key as $k | any($keys | fromjson[]; . == $k)
    )
)

edited Aug 21 '18 at 20:48

answered Apr 08 '15 at 15:03

Jeff Mercado

129,526
32
251
272

score 8 · Answer 4 · answered Apr 11 '17 at 19:02

Here is some additional clarification

For the input object {"key1":1, "key2":2, "key3":3} I would like to drop all keys that are not in the set of desired keys ["key1","key3","key4"]

 jq -n --argjson desired_keys '["key1","key3","key4"]'  \
       --argjson input '{"key1":1, "key2":2, "key3":3}' \
    ' $input
    | with_entries(
          select(
              .key == ($desired_keys[])
          )
       )'

with_entries converts {"key1":1, "key2":2, "key3":3} into the following array of key value pairs and maps the select statement on the array and then turns the resulting array back into an object.

Here is the inner object in the with_entries statement.

[
  {
    "key": "key1",
    "value": 1
  },
  {
    "key": "key2",
    "value": 2
  },
  {
    "key": "key3",
    "value": 3
  }
]

we can then select the keys from this array that meet our criteria.

This is where the magic happens... here is a look at whats going on in the middle of this command. The following command takes the expanded array of values and turns them into a list of objects that we can select from.

jq -cn '{"key":"key1","value":1}, {"key":"key2","value":2}, {"key":"key3","value":3}
      | select(.key == ("key1", "key3", "key4"))'

This will yield the following result

{"key":"key1","value":1}
{"key":"key3","value":3}

The with entries command can be a little tricky but its easy to remember that it takes a filter and is defined as follows

def with_entries(f): to_entries|map(f)|from_entries;

This is the same as

def with_entries(f): [to_entries[] | f] | from_entries;

The other part of the question that confuses people is the multiple matches on the right hand side of the ==

Consider the following command. We see the output is an outer production of all the left hand lists and the right hand lists.

jq -cn '1,2,3| . == (1,1,3)'
true
true
false
false
false
false
false
false
true

If that predicate is in a select statement, we keep the input when the predicate is true. Note you can duplicate the inputs here too.

jq -cn '1,2,3| select(. == (1,1,3))'
1
1
3

peak · Answer 5 · 2022-11-23T18:05:38.843

7

Jeff's answer has a couple of unnecessary inefficiencies, both of which are addressed by the following, on the assumption that --argjson keys is used instead of --arg keys:

with_entries( select( .key as $k | $keys | index($k) ) )

Even better, if your jq has IN:

with_entries(select(.key | IN($keys[])))

edited Nov 23 '22 at 18:05

answered Apr 11 '17 at 19:20

peak

105,803
17
152
177

1

thats clever. avoids going through the entire outer product of input keys and desired keys. – Jon Apr 11 '17 at 20:37

mmoya · Answer 6 · 2023-02-07T11:03:59.853

If you are sure that all keys in the input array are present in the original object, you can use the object construction shortcut.

$ echo '{"1":"a","2":"b","3":"c"}' | jq '{"1", "3"}'
{
  "1": "a",
  "3": "c"
}

Numbers should be quoted to force jq to interpret them as keys instead of literals. In the case of keys not resembling a number, quotes are not needed:

$ echo '{"key1":"a","key2":"b","key3":"c"}' | jq '{key1, key3}'
{
  "key1": "a",
  "key3": "c"
}

Adding a non-existent key will yield a null value, unlikely what OP wanted:

$ echo '{"1":"a","2":"b","3":"c"}' | jq '{"1", "3", "4"}'
{
  "1": "a",
  "3": "c",
  "4": null
}

but those can be filtered out:

$ echo '{"1":"a","2":"b","3":"c"}' | jq '{"1", "3", "4"} | with_entries(select(.value != null))'
{
  "1": "a",
  "3": "c"
}

Although this answer doesn't receive a valid input json array as OP asked, I find it useful for just filtering some keys you know are present.

An example usecase: get aud and iss from a JWT. The following is very succint:

echo "jwt-as-json" | jq '{aud, iss}'

jq: selecting a subset of keys from an object

6 Answers6

Linked