1

I have multiple nested dictionaries of varying depth.

  • A dictionary can contain a list which in itself can contain further nested dictionaries.

  • A dictionary can also contain a "leaf-list" which is simply a list with values.

  • The order of the list elements does not matter.

    base_d = {'level 1' : {'level 2': [{'level 3': [1,2,3], 'level 4': {'level 6': 'replace_me'} } ] } }

update_d = {'level 1' : {'level 2': [{'level 4': {'level 6': 'new_value','level 7':456},'level 5':123  , 'level 3': [1,2,3,4] }] } }

I want to update all values in base_d with the values inside update_d so that the output dictionary becomes

 new_d = {'level 1' : {'level 2': [{'level 3': [1,2,3,4], 'level 4': {'level 6': 'new_value'} } ] } }

I have looked into the pydantic.utils.deep_update library and also tried the following solution inside Update value of a nested dictionary of varying depth

import collections

def update(orig_dict, new_dict):
    for key, val in new_dict.iteritems():
        if isinstance(val, collections.abc.Mapping):
            tmp = update(orig_dict.get(key, { }), val)
            orig_dict[key] = tmp
        elif isinstance(val, list):
            orig_dict[key] = (orig_dict.get(key, []) + val)
        else:
            orig_dict[key] = new_dict[key]
    return orig_dict

But none seem to be able to deal with nested dictionaries inside a list.

  from pydantic.utils import deep_update

  deep_update(base_d,update_d)

  {'level 1': {'level 2': [{'level 4': {'level 6': 'new_value', 'level 7': 456}, 'level 5': 123, 'level 3': [1, 2, 3, 4]}]}}

So how would I be able to update values when I can have lists of nested dictionaries at generic depth inside the base_d and update_d?

Kspr
  • 665
  • 7
  • 19
  • There is a bug in update_d at `'level 3': [1,2,3,4]` which should be a dict: `{'level 3': [1,2,3,4]}` inside the list. – areop-enap Dec 08 '22 at 03:00
  • You also should clarify how lists should be threaded, since lists can contain duplicate items. There are two ways, either both lists are just concatenated or each element address from `base_d[address]` is overwritten with related item from `update_d[address]`. – areop-enap Dec 08 '22 at 04:39

1 Answers1

0

The following function assumes you want to thread lists as dictionaries with item addresses as keys:

['a', 'b', 'c'] --> {0:'a', 1:'b', 2:'c'}

So that all elements of base are updated with those from update:

base_d = {'level 1' : {'level 2': [{'level 3': [1,2,3], 'level 4': {'level 6': 'replace_me'} } ] }, 'level 6': 6 }

update_d = {'level 1' : {'level 2': [{'level 4': {'level 6': 'new_value','level 7':456},'level 5':123 } , {'level 3': [1,2,3,4]} ] }, 'level 7': 7 }

def merge(base, update):
  if isinstance(base, dict):
    return base | update | {k: merge(base[k], v) for k,v in update.items() if k in base}
  elif isinstance(base, list):
    return [merge(b, u) for b, u in zip(base, update)] + update[len(base):] + base[len(update):]
  else:
    return update

print ( merge ( base_d, update_d ) )

Output of this function:

{'level 1': {'level 2': [{'level 3': [1, 2, 3], 'level 4': {'level 6': 'new_value', 'level 7': 456}, 'level 5': 123}, {'level 3': [1, 2, 3, 4]}]}, 'level 6': 6, 'level 7': 7}

Note: the dictionary with 'level 3' is contained twice in the output because it is element 1 in the list of base and element 2 in update's list. I think mixing lists with dictionaries isn't such a good idea in your case because of the behavior of 'level 3'. Such a merge function might lead to unwanted behavior in case the order of lists is different.

I changed the update structure so that 'level 3' is part of the first dictionary inside the list:

update_d = {'level 1' : {'level 2': [{'level 4': {'level 6': 'new_value','level 7':456},'level 5':123 , 'level 3': [1,2,3,4]} ] }, 'level 7': 7 }

Then the output is more usefull:

{'level 1': {'level 2': [{'level 3': [1, 2, 3, 4], 'level 4': {'level 6': 'new_value', 'level 7': 456}, 'level 5': 123}]}, 'level 6': 6, 'level 7': 7}

The entry 'level 3' is contained only once and updated.

areop-enap
  • 396
  • 1
  • 7