Filter object by property and select with key in jmespath

Question

I'm trying to filter properties of an object in jmespath based on the value of a subproperty and want to include only those properties where the subproperty is set to a specific value.

Based on this example data:

{
  "a": {
    "feature": {
      "enabled": true,
    }
  },
  "b": {
  },
  "c": {
    "feature": {
      "enabled": false
     }
  }
}

I'd like to get an object with all properties where the feature is enabled.

{
  "a": {
    "feature": {
      "enabled": true,
    }
  }
}

I figured I could use this jmespath query to filter the objects where property. enabled is set to true. Unfortunateley, it doesn't seem to work and instead returns an empty array.

*[?feature.enabled==`true`]

*.feature.enabled or *[feature.enabled] return just the boolean values without any context.

Even if *[?feature.enabled==true] would work, it would just be an array of the property values, but I need the keys (a and c) aswell. Is there any way to make this happen in jmespath?

This is all part of an ansible playbook, so there would certainly be a way to achieve selection in a different way (Jinja2 templates or custom plugin) but I wanted to try jmespath and would reason, that it should be capable of such a task.

More than query anything, you want to delete certain keys based on nested data. Something like [this question for Perl](http://stackoverflow.com/q/18660309/2947502). I too would like to know if you can do it with JMESPath. — techraf, Jan 11 '17 at 01:21

techraf · Answer 1 · 2018-09-08T08:32:53.360

11

With dict2items filter in Ansible 2.5 and later, you can do it with:

- debug:
    msg: "{{ dict(my_data | dict2items | json_query('[?value.feature.enabled].[key, value]')) }}"

The result:

"msg": {
    "a": {
        "feature": {
            "enabled": true
        }
    }
}

edited Sep 08 '18 at 08:32

answered Sep 08 '18 at 08:16

techraf

64,883
27
193
198

could you please help me in this related question https://stackoverflow.com/questions/59206765/jsonpath-extract-object-meeting-multiple-criteria ? – MasterJoe Dec 06 '19 at 05:05

Konstantin Suvorov · Accepted Answer · 2017-09-21T08:22:35.570

8

Sorry, but AFAIK this is impossible in native JMESPath.
There are custom built-in functions for this purpose in different tools like to_entries in jq.
For jmespath.py and thus for Ansible there is hanging pull request to implement keys manipulation.

Update: I've made a patched version of json_query filter.
See this answer for additional info.

edited Sep 21 '17 at 08:22

answered Jan 11 '17 at 07:20

Konstantin Suvorov

65,183
9
162
193

Thanks, I'm following that pull request and for the time being I have solved this problem with a jinja2 loop to select matching dictionary items and put them together to a new dict. – Johannes Müller Jan 23 '17 at 17:34
Looks like a lot of online json viewing/formatting tools use jmespath for querying. Do you know any good tool which uses jsonpath instead ? – MasterJoe Dec 05 '19 at 18:46
could you please help me in this related question https://stackoverflow.com/questions/59206765/jsonpath-extract-object-meeting-multiple-criteria ? – MasterJoe Dec 06 '19 at 05:05

dreftymac · Answer 3 · 2019-03-23T06:50:51.730

Short answer (TL;DR)

Actually, yes, this is possible with nothing more than native jmespath
The problem is, queries against the source dataset will be extremely cumbersome, because the source dataset is poorly normalized for this kind of general-purpose jmespath query.

Example

The following (way-too-long) jmespath query against the source data in the OP...

[
  {
      "item_key":           `a`
      ,"feature_enabled":   @.a.feature.enabled
      ,source_object:       @.a
  }
  ,{
      "item_key":           `b`
      ,"feature_enabled":   @.b.feature.enabled
      ,source_object:       @.b
  }
  ,{
      "item_key":           `c`
      ,"feature_enabled":   @.c.feature.enabled
      ,source_object:       @.c
  }
]|[? feature_enabled == `true`]

... produces the following result

[
  {
    "item_key": "a",
    "feature_enabled": true,
    "source_object": {
      "feature": {
        "enabled": true
      }
    }
  }
]

Which is identical or substantially similar to the desired output, but the fact that we had to bend our brain to get there suggests we are trying to force a square peg through a round hole.

Pitfalls

The reason this jmespath query looks so long and cumbersome is that the source dataset itself is poorly normalized for a general purpose jmespath query.

That is because it uses object keys as a top-level collation method, when a sequentially-indexed-list would have sufficed.

Whenever you have a dataset that can potentially contain an arbitrary number of values it is almost always preferable to use a sequence for top-level collation, instead of object keys.

If you find you can do something in jmespath, but you have to modify your jmespath query whenever you add another "entry" to your "set of entries of arbitrary (non-fixed) length" you are fighting against Jmespath instead of working with it.

Whenever you see a query that seems "impossible to accomplish" with Jmespath, you are almost certainly dealing with a data structure that is using objects where sequences may have been more suitable.

Object keys usually mean a fixed number of properties, which jmespath can handle just fine.

Even object properties of arbitrarily deep nesting are just fine, so long as those object properties are not being used as a substitute for sequential enumeration.

Things only start to get uncomfortable when you find you are having to create sequences-of-objects in order to get around objects-of-objects ... which is entirely doable in jmespath, but it is going to be painful.

Looks like a lot of online json viewing/formatting tools use jmespath for querying. Do you know any good tool which uses jsonpath instead ? — MasterJoe, Dec 05 '19 at 18:45
For example, look at this chart: [Sample comparison](https://raw.githubusercontent.com/dreftymac/trypublic/master/lab2019/jmespath-comparison/capture-uu342rofa88smur.png) — dreftymac, Dec 05 '19 at 20:16
So, I wonder if it is better to just use some programming language instead of any json query language. JMES looks ugly. — MasterJoe, Dec 06 '19 at 03:53
could you please help me in this related question https://stackoverflow.com/questions/59206765/jsonpath-extract-object-meeting-multiple-criteria ? — MasterJoe, Dec 06 '19 at 05:05

score 1 · Answer 4 · edited Feb 09 '22 at 13:43

1

You can use:

<json_data>|[*]|[? @.feature.enabled]

edited Feb 09 '22 at 13:43

blackgreen

34,072
23
111
129

answered Feb 09 '22 at 13:40

user9642565

21
2

3

What's this do? Why do you prefer this over the existing answers, which have been validated by the community? – Jeremy Caney Feb 12 '22 at 00:57
The issue using `*` on a dict is that you are loosing the keys of it, and that is one of the requirement of the OP to still have the original keys structure. – β.εηοιτ.βε Feb 12 '22 at 20:40
While this code may solve the question, [including an explanation](//meta.stackexchange.com/q/114762) of how and why this solves the problem would really help to improve the quality of your post, and probably result in more up-votes. Remember that you are answering the question for readers in the future, not just the person asking now. Please [edit] your answer to add explanations and give an indication of what limitations and assumptions apply. – Yunnosch Apr 07 '22 at 21:48

Filter object by property and select with key in jmespath

4 Answers4

Short answer (TL;DR)

Example

Pitfalls

See also

Linked