8

I have a dictionary of packages with package-name being the key and a dictionary of some details being the value:

{
        "php7.1-readline": {
            "latest": "7.1.9-1+ubuntu14.04.1+deb.sury.org+1", 
            "origins": [
                "ppa.launchpad.net"
            ], 
            "version": "7.1.6-2~ubuntu14.04.1+deb.sury.org+1", 
            "www": "http://www.php.net/"
        }, 
        "php7.1-xml": {
            "latest": "7.1.9-1+ubuntu14.04.1+deb.sury.org+1", 
            "origins": [
                "ppa.launchpad.net"
            ], 
            "version": "7.1.6-2~ubuntu14.04.1+deb.sury.org+1", 
            "www": "http://www.php.net/"
        }, 
        "plymouth": {
            "version": "0.8.8-0ubuntu17.1"
        },
    ....
}

I'd like to reduce the above to a dictionary with only the packages, that have the latest-attribute in their values.

It would seem like json_query is the filter to use, but I can't figure out the syntax. The examples out there all seem to operate on lists of dictionaries, not dictionaries of same...

For example, if I "pipe" the above dictionary into json_query('*.latest'), I get the list of the actual latest versions:

[
  "7.1.9-1+ubuntu14.04.1+deb.sury.org+1",
  "7.1.9-1+ubuntu14.04.1+deb.sury.org+1",
  "7.1.6-2~ubuntu14.04.1+deb.sury.org+1"
]

How can I get the entire dictionary-elements instead?

Any hope?

Mikhail T.
  • 3,043
  • 3
  • 29
  • 46

3 Answers3

4

You can't perform this translation (I think) exclusively with Jinja filters, but you can get there by applying a little Ansible logic as well. The following playbook uses a with_dict loop to loop over the items in your dictionary, and build a new dictionary from matching ones:

- hosts: localhost                                                              
  vars:                                                                         
    packages: {                                                                 
        "php7.1-readline": {                                                    
          "latest": "7.1.9-1+ubuntu14.04.1+deb.sury.org+1",                     
          "origins": [                                                          
            "ppa.launchpad.net"                                                 
          ],                                                                    
          "version": "7.1.6-2~ubuntu14.04.1+deb.sury.org+1",                    
          "www": "http://www.php.net/"                                          
        },                                                                      
        "php7.1-xml": {                                                         
          "latest": "7.1.9-1+ubuntu14.04.1+deb.sury.org+1",                     
          "origins": [                                                          
            "ppa.launchpad.net"                                                 
          ],                                                                    
          "version": "7.1.6-2~ubuntu14.04.1+deb.sury.org+1",                    
          "www": "http://www.php.net/"                                          
        },                                                                      
        "plymouth": {                                                           
          "version": "0.8.8-0ubuntu17.1"                                        
        }                                                                       
      }                                                                         

  tasks:                                                                        
    - set_fact:                                                                 
        new_packages: >                                                         
          {{ new_packages|default({})|                                          
                combine({item.key: item.value}) }}                              
      with_dict: "{{ packages }}"                                               
      when: "{{ item.value.latest is defined }}"                                

    - debug:                                                                    
        var: new_packages
larsks
  • 277,717
  • 41
  • 399
  • 399
  • Thanks, I actually tried something like this. However, the `when`-clause lists _each_ package it skips -- hundreds of lines of output because, of course, I need to process _all_ packages installed on each system, filtering those, for which updates are available... – Mikhail T. Sep 19 '17 at 19:39
  • So, all you care about is the resulting variable, `new_packages`. The number of lines of output doesn't matter. That's just informative. – larsks Sep 19 '17 at 23:34
4

With dict2items filter added in December 2017, it is possible using native functionality:

- debug:
    msg: "{{ dict(pkg | dict2items | json_query('[?value.latest].[key, value.latest]')) }}"

The result:

"msg": {
    "php7.1-readline": "7.1.9-1+ubuntu14.04.1+deb.sury.org+1",
    "php7.1-xml": "7.1.9-1+ubuntu14.04.1+deb.sury.org+1"
}
techraf
  • 64,883
  • 27
  • 193
  • 198
  • dict2items supported startind from ansible 2.6 (https://docs.ansible.com/ansible/latest/user_guide/playbooks_filters.html#dict-filter) – mmv-ru Aug 10 '19 at 17:44
2

You are correct to link this question to https://stackoverflow.com/a/41584889/2795592.

There are no options to manipulate keys and values simultaneously with json_query out of the box (as of Ansible 2.4.0).

Here's patched json_query.py that supports jq-like to_entries/from_entries functions. You can put it into ./filter_plugins near your playbook and make this query:

- debug:
    msg: "{{ pkg | json_query('to_entries(@) | [?value.latest].{key:key, value:value.latest} | from_entries(@)')}}"

to get this result:

"msg": {
    "php7.1-readline": "7.1.9-1+ubuntu14.04.1+deb.sury.org+1",
    "php7.1-xml": "7.1.9-1+ubuntu14.04.1+deb.sury.org+1"
}

I'll make PR to ansible as soon as I have some spare time.

Konstantin Suvorov
  • 65,183
  • 9
  • 162
  • 193
  • Thank you, but it is kind of sad... I deliberately picked the dictionary structure for the packages so as to be able to lookup a package's details by name quickly (`O(ln(n))`). Converting it to a list makes everything linear... As long as we are patching things anyway, can we not imagine something closer to Ansible's `with_dict` -- where you can operate on each `item`'s `key` and `value` directly? – Mikhail T. Sep 21 '17 at 15:44
  • You a free to modify my gist example. Or write your custom plugin to process your specific dataset very fast. Or ask some Ansible guru to make it :-D – Konstantin Suvorov Sep 21 '17 at 15:51
  • I know, I can write a custom filter. Have, in fact. But I was hoping, it could be done without one -- with some clever combination of `extract`, `map`, `selectattr`, and/or `json_query`... Oh, well, thank you for your help again -- a negative result is still a result, is not it... – Mikhail T. Sep 21 '17 at 15:59
  • So I created my own little filter to emulate `to_entries` -- and am feeding its result into the `json_query`. But some of the packages in my list have `.security`-attribute _instead of_ (or in addition to) `.latest`. Can jmespath do something like `[?value.latest].{package:key, version:value.latest, security:'False'} || [?value.security].{package:key, version:value.security, security:'True'}` – Mikhail T. Sep 21 '17 at 17:55