Python: Extracting key-pair values from nested dictionary in yaml_file1 and putting a key-pair in yaml_file2

Question

yaml_file1.yaml

container:
 name: NAME1
 number: 222
type:
  compression:lz
  function: boot_a
  type_revno:REV123
  type_prodno: REV345
  fw:
     prodno:ABC123
     revno:ABC345
     prodno:DEF123
     revno:DEF345
  url:"file:///path1"
  slot : 1
 
  compression:zip
  function: boot_b
  type_revno:REV111
  type_prodno: REV222
  fw:
     prodno:XYZ111
     revno:XYZ222
     prodno: UVW111
     revno:UVW222
  url: "file:///path2"
  slot : 2
  
  compression:lz
  function: appl_a
  type_revno:REV333
  type_prodno: REV444
  fw:
     prodno:XYZ333
     revno:XYZ444
     prodno: UVW333
     revno:UVW444
  url: "file:///path3"
  slot : 3
  
 compression:zip
 function: appl_b
 type_revno:REV333
 type_prodno: REV444
 type_revno:REV333
 type_prodno: REV444
 fw:
    prodno:XYZ333
    revno:XYZ444
    prodno: UVW333
    revno:UVW444
 url: "file:///path4"
  slot : 4

   
*********************************
yaml_file2.yaml

container:
 name: NAME1
 number: 222
 timestamp: 1634495717
type:
  compression:lz
  function: boot_a
  type_revno:REV123
  type_prodno: REV345
  fw:
     prodno:ABC123
     revno:ABC345
     prodno:DEF123
     revno:DEF345
 offset: 0
 md5sum : (some md5sum value)
 size: 3000
 slot : 1
 
 compression:zip
 function:boot_b
 type_revno:REV111
 type_prodno: REV222
 fw:
     prodno:XYZ111
     revno:XYZ222
     prodno: UVW111
     revno:UVW222
 offset : 1
 md5sum : (some md5sum value)
 size: 2000
 slot : 2
 
 compression:lz
 function: appl_a
 type_revno:REV111
 type_prodno: REV222
 fw:
     prodno:XYZ111
     revno:XYZ222
     prodno: UVW111
     revno:UVW222
 offset: 2
 md5sum : (some md5sum value)
 size : 1000
 slot : 3
  
 compression:zip
 function: appl_b
 type_revno:REV333
 type_prodno: REV444
 type_revno:REV333
 type_prodno: REV444
 fw:
    prodno:XYZ333
    revno:XYZ444
    prodno: UVW333
    revno:UVW444
 offset: 3
 md5sum : (some md5sum value)
 slot : 4

As we can see, yaml_file2.yaml has url parameter missing for all the instances of 'type'. I want to pick up 'url' parameter and its value from yaml_file1.yaml and put it for corresponding instances of 'type' in yaml_file2.yaml. We can differentiate the 'type' from one another using 'function' names for mapping.

Can we do this in python using dictionary methods?

I am an amateur and still learning. So far I have only understood about accessing non-nested dictionary.

If I try to access urls like this:

with open('yaml_file1.yaml','r') as ingen:
    datagen = yaml.full_load(ingen)

for k, v in datagen["type"].items():
    print (datagen["type"][k]["url"])

I get

Traceback (most recent call last):
  File "compareyaml.py", line 43, in <module>
    for k, v in datagen["type"].items():
AttributeError: 'list' object has no attribute 'items'

Can someone please help?

score 0 · Answer 1 · answered Nov 10 '21 at 21:56

You can start by reading the yaml files into dictionaries in your name space. You can use the yaml package in python for this:

import yaml

with open(r'yaml_file1.yaml') as file:
    dict1 = yaml.load(file, Loader=yaml.FullLoader)
with open(r'yaml_file2.yaml') as file:
    dict2 = yaml.load(file, Loader=yaml.FullLoader)

Now you have got two dictionaries that you play with as you please. You get the wanted value from dict1 and insert it to dict2.

and at the end you write back the dictionary to yaml file:

with open(r'result.yaml', 'w') as file:
    res = yaml.dump(dict2, file)

Thanks Abdelaziz. That part I have done...it's the play with dictionaries part I'm looking for guidance on. I'm trying to use dictionary methods get(), items() for this. dict1.keys outputs: dict1_keys(['container', 'type']) dict2.items outputs all key-valuue pairs in dictionary format — Kaytoo, Nov 10 '21 at 22:08

score 0 · Answer 2 · edited Nov 11 '21 at 22:07

Yaml file is stored as the 1 dictionary and 1 list.

Dictionary : container List : type

The list 'type' will have nested dictionaries for every type.

Figured how to access url and add to dict2:

url_var0 = datagen["type"][0]["url"]
url_var1 = datagen["type"][1]["url"]
url_var2 = datagen["type"][2]["url"]
url_var3 = datagen["type"][3]["url"]

dict2["type"][0]["url"] = url_var0
dict2["type"][1]["url"] = url_var1

and so on.

My remaining challenge is to figure out how to write a for loop for this.

Will update here once I figure out.

score 0 · Answer 3 · answered Nov 11 '21 at 22:41

Woohoo! Figured out!

for ctr, idx in zip(datagen["type"], dict2["type"]):
    idx['url'] = ctr['url']

Before reaching here, I wrote 2 for loops, nested, separate...tried storing/assigning index. Nothing worked. I was thinking like a C programmer. Stumbled upon : Iterating through 2 dictionaries simulataneously

I know for Python experts this might be a small thing. But discovering 'zip' to run through 2 dictionaries at the same time was a feat for me, being an amateur.

Thank you!

Python: Extracting key-pair values from nested dictionary in yaml_file1 and putting a key-pair in yaml_file2

3 Answers3