282

I'm looking for a way to update dict dictionary1 with the contents of dict update wihout overwriting levelA

dictionary1 = {
    "level1": {
        "level2": {"levelA": 0, "levelB": 1}
    }
}
update = {
    "level1": {
        "level2": {"levelB": 10}
    }
}
dictionary1.update(update)
print(dictionary1)
{
    "level1": {
        "level2": {"levelB": 10}
    }
}

I know that update deletes the values in level2 because it's updating the lowest key level1.

How could I tackle this, given that dictionary1 and update can have any length?

Nico Schlömer
  • 53,797
  • 27
  • 201
  • 249
jay_t
  • 3,593
  • 4
  • 22
  • 27
  • 2
    Is the nesting always three levels deep or can you have nesting of an arbitrary depth? – ChristopheD Jul 12 '10 at 23:03
  • 3
    It can have any depth/length. – jay_t Jul 13 '10 at 07:55
  • 2
    Correct me if I’m wrong but it seems like the ideal solution here requires implementation of the composite design pattern. – Alexander McNulty Mar 21 '19 at 18:34
  • In the general case this may not be well defined unless you have the update take additional arguments[s] specifying key paths to preserve. A possible interesting sort of deep update is a merge_update preserving what was unique in the dictts at all depths and replacing or updating what is not recursively. – Samantha Atkins Jun 30 '22 at 00:51

31 Answers31

376

@FM's answer has the right general idea, i.e. a recursive solution, but somewhat peculiar coding and at least one bug. I'd recommend, instead:

Python 2:

import collections

def update(d, u):
    for k, v in u.iteritems():
        if isinstance(v, collections.Mapping):
            d[k] = update(d.get(k, {}), v)
        else:
            d[k] = v
    return d

Python 3:

import collections.abc

def update(d, u):
    for k, v in u.items():
        if isinstance(v, collections.abc.Mapping):
            d[k] = update(d.get(k, {}), v)
        else:
            d[k] = v
    return d

The bug shows up when the "update" has a k, v item where v is a dict and k is not originally a key in the dictionary being updated -- @FM's code "skips" this part of the update (because it performs it on an empty new dict which isn't saved or returned anywhere, just lost when the recursive call returns).

My other changes are minor: there is no reason for the if/else construct when .get does the same job faster and cleaner, and isinstance is best applied to abstract base classes (not concrete ones) for generality.

Alex Telon
  • 1,107
  • 1
  • 14
  • 30
Alex Martelli
  • 854,459
  • 170
  • 1,222
  • 1,395
  • 11
    +1 Good catch on the bug -- doh! I figured someone would would have a better way to handle the `isinstance` test, but thought I'd take a stab at it. – FMc Jul 13 '10 at 02:58
  • @jay_t. you're welcome -- yep, I agree that the collections' module abstract base classes (Mapping etc), which were new in Python 2.6, are really nice (you can also make your own ABCs with module abc!-). – Alex Martelli Jul 13 '10 at 23:41
  • For me this only works when using `dict` instead of `collections.Mapping`?! Otherwise `isinstance` always returns false. (Python 2.5.4) – fuenfundachtzig Jan 07 '11 at 10:12
  • adding a depth arg may be useful (though less elegant): `def update(d, u, depth=-1):` ... `if not depth==0 and isinstance(v, collections.Mapping):` ... `r = update(d.get(k, {}), v, depth=max(depth-1,-1))` ... – hobs Dec 26 '12 at 22:08
  • 9
    Another minor "feature" causes this to raise `TypeError: 'int' object does not support item assignment.` when you, e.g. `update({'k1': 1}, {'k1': {'k2': 2}})`. To change this behavior, and instead expand the depth of dictionaries to make room for deeper dictionaries you can add an `elif isinstance(d, Mapping):` around the `d[k] = u[k]` and after the `isinstance` condition. You'll also need to add an `else: d = {k: u[k]}` to deal with the case that the updating dict is deeper than the original dict. Happy to edit the answer, but don't want to dirty concise code that solves the OP's problem. – hobs Dec 27 '12 at 01:10
  • 3
    Why use `isinstance(v, collections.Mapping)` rather than `isinstance(v, dict)`? In the event that OP decides to start using collections? – Matt Feb 08 '13 at 17:07
  • 3
    @Matt Yea, or any other mapping-derived object (lists of pairs of things). Makes the function more general and less likely to quietly ignore mapping-derived objects and leave them un-updated (insidious error that the OP might not ever see/catch). You almost always want to use Mapping to find dict types and basestring to find str types. – hobs Feb 08 '13 at 22:39
  • 4
    Recursion is only needed if the old and new value are both collections: `if isinstance(d.get(k, None), collections.Mapping) and isinstance(v, collections.Mapping): d[k] = update(d[k], v)` followed by `else: d[k] = v` – Monica For CEO Dec 29 '14 at 22:05
  • 3
    If you're running this under Python 3+ change `u.iteritems()` to `u.items()`, otherwise you will encounter: `AttributeError: 'dict' object has no attribute 'iteritems'` – Greg K Dec 30 '14 at 16:28
  • 2
    Note that `update({'k1': None}, {'k1': {'foo': 'bar'})` causes a crash, whereas what I'd expect is `{'k1': {'foo': 'bar'}`. – paulmelnikow Jan 22 '15 at 00:47
  • 2
    @paulmelnikow, the problem of replacing integers with dictionaries should be addressed by my answer below – bscan Sep 02 '15 at 15:24
  • Thx. I somewhat like immutability of the original dict, so I added the line: final_dict = copy.deepcopy(d) – cecemel Jun 16 '16 at 12:46
  • Maybe you could add a third option with six.iteritems(u) so that your code is 2/3 compatible – Charlesthk Jan 25 '17 at 19:34
  • .get is cleaner, more pythonic and the better choice in general but it is NOT faster. It is slow as hell in comparison if the default case happens a reasonable amount. – Ramon Nov 22 '17 at 06:55
  • Cross referencing a similar function from the MochiKit JavaScript libary (MochiKit is heavily inspired by Python): https://mochi.github.io/mochikit/doc/html/MochiKit/Base.html#fn-updatetree I think `updatetree` is a great name for this function. – EoghanM Oct 17 '18 at 11:26
  • [My answer](https://stackoverflow.com/a/60321833/4653485), inspired from this one, adds a test to handle the `update({'k1': None}, {'k1': {'foo': 'bar'})` case. – Jérôme Feb 20 '20 at 14:15
  • 2
    @hobs I think it will be most easy if we just replace line: `d[k] = update(d.get(k, {}), v)` with: `d[k] = update({}, v)`. That is, pass an **empty dictionary** that will be populated at next round with new data. This must fix a problems in the comments above. – Sultan Aug 22 '20 at 16:19
  • @Sultan Wow! That looks like magic. I haven't tested that update() approach. But if you have tested it thoroughly, feel free to edit the answer directly. Or you can show me your tests and I can make the edit if you don't want to. – hobs Aug 22 '20 at 20:47
  • 2
    @hobs I posted the solution as a new answer, as editing this answer is not allowed. I've already tested the code. It works great. Enjoy) https://stackoverflow.com/a/63543967/1999801 – Sultan Aug 23 '20 at 11:45
  • What about some example? – Pithikos Jan 13 '21 at 17:11
  • if you replace that line with `d[k] = update(d.get(k) or {}, v)` that prevents the crash mentioned by @paulmelnikow . It works becasue `get()`will by default return `None`. So if you move the `{}`out of the get and instead use it with `or` in the situtation of the crash it will execute as `None or {}` and that works fine. – Nico Knoll Jan 28 '21 at 18:29
  • Thanks a lot for this amazing solution! A modified version of this code with some extra features (i.e. keyword updating, fixed dictionary) can be found in [this gist](https://gist.github.com/rickstaa/1e0f4ebcc7c35bdee2cae9ccf6fec6ec). – rickstaa Jun 10 '21 at 08:54
  • You should use `MutableMapping`, since there are immutable `Mapping`s, e.g. `types.MappingProxyType`, which would fail on the next step. – wjandrea Mar 31 '23 at 23:47
52

If you happen to be using pydantic (great lib, BTW), you can use one of its utility methods:

from pydantic.utils import deep_update

dictionary1 = deep_update(dictionary1, update)

UPDATE: reference to code, as pointed by @Jorgu. If installing pydantic is not desired, the code is short enough to be copied, provided adequate licenses compatibilities.

kepler
  • 1,712
  • 17
  • 18
  • 8
    This should be upvoted. Most people should be using this now. No need to bake your own implementation of this – Shaun Dec 07 '21 at 13:31
  • 3
    [pydantic.utils.deep_update on Github](https://github.com/samuelcolvin/pydantic/blob/9d631a3429a66f30742c1a52c94ac18ec6ba848d/pydantic/utils.py#L198) TLDR: it's recursive, typed, and accept several updates at the same time – Jorgu Jan 26 '22 at 16:09
  • but pydantic is a huge library and in many cases its not feasible to waste that much space :( – Piakkaa May 03 '22 at 06:13
  • @Piakkaa No need to import the entirety of `pydantic`, by using `from pydantic.utils import deep_update` the code is surgically getting a small portion of the module. – ingyhere Jul 25 '23 at 00:56
  • is it possible to install only some part of pydantic using pip? – Piakkaa Jul 31 '23 at 07:00
35

Took me a little bit on this one, but thanks to @Alex's post, he filled in the gap I was missing. However, I came across an issue if a value within the recursive dict happens to be a list, so I thought I'd share, and extend his answer.

import collections

def update(orig_dict, new_dict):
    for key, val in new_dict.iteritems():
        if isinstance(val, collections.Mapping):
            tmp = update(orig_dict.get(key, { }), val)
            orig_dict[key] = tmp
        elif isinstance(val, list):
            orig_dict[key] = (orig_dict.get(key, []) + val)
        else:
            orig_dict[key] = new_dict[key]
    return orig_dict
Nate Glenn
  • 6,455
  • 8
  • 52
  • 95
  • 4
    I think this should probably be (to be a bit safer): `orig_dict.get(key, []) + val`. – Andy Hayden Oct 15 '14 at 06:31
  • 3
    Since dicts are mutable, you are changing the instance you are passing as argument. Then, you don't need to return orig_dict. – gabrielhpugliese Feb 27 '15 at 19:33
  • 4
    I think most people would expect the definition to return the updated dict even though it is updated in place. – Kel Solaar Jun 12 '15 at 20:56
  • The default logic in the onosendi's code is to append updated list to the original list. If you need to update overwrite the original list, you need to set orig_dict[key]=val – intijk Nov 01 '16 at 18:00
  • 2
    @gabrielhpugliese returning the original is needed if called with a dictionary literal, e.g. `merged_tree = update({'default': {'initialvalue': 1}}, other_tree)` – EoghanM Oct 17 '18 at 11:24
34

Same solution as the accepted one, but clearer variable naming, docstring, and fixed a bug where {} as a value would not override.

import collections


def deep_update(source, overrides):
    """
    Update a nested dictionary or similar mapping.
    Modify ``source`` in place.
    """
    for key, value in overrides.iteritems():
        if isinstance(value, collections.Mapping) and value:
            returned = deep_update(source.get(key, {}), value)
            source[key] = returned
        else:
            source[key] = overrides[key]
    return source

Here are a few test cases:

def test_deep_update():
    source = {'hello1': 1}
    overrides = {'hello2': 2}
    deep_update(source, overrides)
    assert source == {'hello1': 1, 'hello2': 2}

    source = {'hello': 'to_override'}
    overrides = {'hello': 'over'}
    deep_update(source, overrides)
    assert source == {'hello': 'over'}

    source = {'hello': {'value': 'to_override', 'no_change': 1}}
    overrides = {'hello': {'value': 'over'}}
    deep_update(source, overrides)
    assert source == {'hello': {'value': 'over', 'no_change': 1}}

    source = {'hello': {'value': 'to_override', 'no_change': 1}}
    overrides = {'hello': {'value': {}}}
    deep_update(source, overrides)
    assert source == {'hello': {'value': {}, 'no_change': 1}}

    source = {'hello': {'value': {}, 'no_change': 1}}
    overrides = {'hello': {'value': 2}}
    deep_update(source, overrides)
    assert source == {'hello': {'value': 2, 'no_change': 1}}

This functions is available in the charlatan package, in charlatan.utils.

surj
  • 4,706
  • 2
  • 25
  • 34
charlax
  • 25,125
  • 19
  • 60
  • 71
  • 5
    Lovely. But had to update `overrides.iteritems()` to `overrides.items()` and `collections.Mapping` to `collections.abc.Mapping` on Python 3.9+ – drstevok Jan 01 '22 at 16:35
22

@Alex's answer is good, but doesn't work when replacing an element such as an integer with a dictionary, such as update({'foo':0},{'foo':{'bar':1}}). This update addresses it:

import collections
def update(d, u):
    for k, v in u.iteritems():
        if isinstance(d, collections.Mapping):
            if isinstance(v, collections.Mapping):
                r = update(d.get(k, {}), v)
                d[k] = r
            else:
                d[k] = u[k]
        else:
            d = {k: u[k]}
    return d

update({'k1': 1}, {'k1': {'k2': {'k3': 3}}})
bscan
  • 2,816
  • 1
  • 16
  • 16
  • I see. You made my `elif` check of the original object type an "enclosing" conditional containing the checks of both the value and the key of that dict/mapping. Clever. – hobs Sep 06 '15 at 19:14
  • This won't work if the inner dict has more than one key. – Wlerin Feb 25 '17 at 23:51
  • @Wlerin , it still works; d will have become a Mapping by that point. Here's a test case with multiple keys: `update({'A1': 1, 'A2':2}, {'A1': {'B1': {'C1': 3, 'C2':4}, 'B2':2}, 'A3':5})`. Do you have an example that doesn't do what you want? – bscan Feb 27 '17 at 15:59
  • Why test `if isinstance(d, collections.Mapping)` on evey iteration? See [my answer](https://stackoverflow.com/a/60321833/4653485). – Jérôme Feb 20 '20 at 14:11
17

Here's an immutable version of recursive dictionary merge in case someone needs it.

Based upon @Alex Martelli's answer.

Python 3.x:

import collections
from copy import deepcopy


def merge(dict1, dict2):
    ''' Return a new dictionary by merging two dictionaries recursively. '''

    result = deepcopy(dict1)

    for key, value in dict2.items():
        if isinstance(value, collections.Mapping):
            result[key] = merge(result.get(key, {}), value)
        else:
            result[key] = deepcopy(dict2[key])

    return result

Python 2.x:

import collections
from copy import deepcopy


def merge(dict1, dict2):
    ''' Return a new dictionary by merging two dictionaries recursively. '''

    result = deepcopy(dict1)

    for key, value in dict2.iteritems():
        if isinstance(value, collections.Mapping):
            result[key] = merge(result.get(key, {}), value)
        else:
            result[key] = deepcopy(dict2[key])

    return result
kabirbaidhya
  • 3,264
  • 3
  • 34
  • 59
9

This question is old, but I landed here when searching for a "deep merge" solution. The answers above inspired what follows. I ended up writing my own because there were bugs in all the versions I tested. The critical point missed was, at some arbitrary depth of the two input dicts, for some key, k, the decision tree when d[k] or u[k] is not a dict was faulty.

Also, this solution does not require recursion, which is more symmetric with how dict.update() works, and returns None.

import collections
def deep_merge(d, u):
   """Do a deep merge of one dict into another.

   This will update d with values in u, but will not delete keys in d
   not found in u at some arbitrary depth of d. That is, u is deeply
   merged into d.

   Args -
     d, u: dicts

   Note: this is destructive to d, but not u.

   Returns: None
   """
   stack = [(d,u)]
   while stack:
      d,u = stack.pop(0)
      for k,v in u.items():
         if not isinstance(v, collections.Mapping):
            # u[k] is not a dict, nothing to merge, so just set it,
            # regardless if d[k] *was* a dict
            d[k] = v

        else:
            # note: u[k] is a dict
            if k not in d:
                # add new key into d
                d[k] = v
            elif not isinstance(d[k], collections.Mapping):
                # d[k] is not a dict, so just set it to u[k],
                # overriding whatever it was
                d[k] = v
            else:
                # both d[k] and u[k] are dicts, push them on the stack
                # to merge
                stack.append((d[k], v))
matusko
  • 3,487
  • 3
  • 20
  • 31
djpinne
  • 673
  • 1
  • 6
  • 9
9

Just use python-benedict (I did it), it has a merge (deepupdate) utility method and many others. It works with python 2 / python 3 and it is well tested.

from benedict import benedict

dictionary1=benedict({'level1':{'level2':{'levelA':0,'levelB':1}}})
update={'level1':{'level2':{'levelB':10}}}
dictionary1.merge(update)
print(dictionary1)
# >> {'level1':{'level2':{'levelA':0,'levelB':10}}}

Installation: pip install python-benedict

Documentation: https://github.com/fabiocaccamo/python-benedict

Note: I am the author of this project

Martijn Pieters
  • 1,048,767
  • 296
  • 4,058
  • 3,343
Fabio Caccamo
  • 1,871
  • 19
  • 21
6

Minor improvements to @Alex's answer that enables updating of dictionaries of differing depths as well as limiting the depth that the update dives into the original nested dictionary (but the updating dictionary depth is not limited). Only a few cases have been tested:

def update(d, u, depth=-1):
    """
    Recursively merge or update dict-like objects. 
    >>> update({'k1': {'k2': 2}}, {'k1': {'k2': {'k3': 3}}, 'k4': 4})
    {'k1': {'k2': {'k3': 3}}, 'k4': 4}
    """

    for k, v in u.iteritems():
        if isinstance(v, Mapping) and not depth == 0:
            r = update(d.get(k, {}), v, depth=max(depth - 1, -1))
            d[k] = r
        elif isinstance(d, Mapping):
            d[k] = u[k]
        else:
            d = {k: u[k]}
    return d
Community
  • 1
  • 1
hobs
  • 18,473
  • 10
  • 83
  • 106
  • 2
    Thanks for this! What use-case might the depth parameter apply to? – Matt Feb 08 '13 at 17:54
  • @Matt when you have some objects/dicts at a known depth that you don't want merged/updated, just overwritten with new objects (like replacing a dict with a string or float or whatever, deep in your dict) – hobs Feb 08 '13 at 22:30
  • 2
    This only works if the update is at most 1 level deeper than the original. For example, this fails: `update({'k1': 1}, {'k1': {'k2': {'k3': 3}}})` I added an answer that addresses this – bscan Sep 02 '15 at 15:32
  • @bscan good catch! never thought of that use case. I guess I should recurse deeper in the elif branches. Any ideas? – hobs Sep 03 '15 at 18:29
  • 2
    Why test `if isinstance(d, Mapping)` on evey iteration? See [my answer](https://stackoverflow.com/a/60321833/4653485). (Also, I'm not sure about your `d = {k: u[k]}`) – Jérôme Feb 20 '20 at 14:13
  • 2
    I was using hobs' answer but ran into the case where the update dict was a lot deeper than the original, Jerome's answer did the trick for me! – Erwan Leroy Nov 23 '21 at 01:43
5

The code below should solve the update({'k1': 1}, {'k1': {'k2': 2}}) issue in @Alex Martelli's answer the right way.

def deepupdate(original, update):
    """Recursively update a dict.

    Subdict's won't be overwritten but also updated.
    """
    if not isinstance(original, abc.Mapping):
        return update
    for key, value in update.items():
        if isinstance(value, abc.Mapping):
            original[key] = deepupdate(original.get(key, {}), value)
        else:
            original[key] = value
    return original
Jérôme
  • 13,328
  • 7
  • 56
  • 106
4

I used the solution @Alex Martelli suggests, but it fails

TypeError 'bool' object does not support item assignment

when the two dictionaries differ in data type at some level.

In case at the same level the element of dictionary d is just a scalar (ie. Bool) while the element of dictionary u is still dictionary the reassignment fails as no dictionary assignment is possible into scalar (like True[k]).

One added condition fixes that:

from collections import Mapping

def update_deep(d, u):
    for k, v in u.items():
        # this condition handles the problem
        if not isinstance(d, Mapping):
            d = u
        elif isinstance(v, Mapping):
            r = update_deep(d.get(k, {}), v)
            d[k] = r
        else:
            d[k] = u[k]

    return d
helvete
  • 2,455
  • 13
  • 33
  • 37
3

It could be that you stumble over a non-standard-dictionary, like me today, which has no iteritems-Attribute. In this case it's easy to interpret this type of dictionary as a standard-dictionary. E.g.: Python 2.7:

    import collections
    def update(orig_dict, new_dict):
        for key, val in dict(new_dict).iteritems():
            if isinstance(val, collections.Mapping):
                tmp = update(orig_dict.get(key, { }), val)
                orig_dict[key] = tmp
            elif isinstance(val, list):
                orig_dict[key] = (orig_dict[key] + val)
            else:
                orig_dict[key] = new_dict[key]
        return orig_dict

    import multiprocessing
    d=multiprocessing.Manager().dict({'sample':'data'})
    u={'other': 1234}

    x=update(d, u)
    x.items()

Python 3.8:

    def update(orig_dict, new_dict):
        orig_dict=dict(orig_dict)
        for key, val in dict(new_dict).items():
            if isinstance(val, collections.abc.Mapping):
                tmp = update(orig_dict.get(key, { }), val)
                orig_dict[key] = tmp
            elif isinstance(val, list):
                orig_dict[key] = (orig_dict[key] + val)
            else:
                orig_dict[key] = new_dict[key]
        return orig_dict

    import collections
    import multiprocessing
    d=multiprocessing.Manager().dict({'sample':'data'})
    u={'other': 1234, "deeper": {'very': 'deep'}}

    x=update(d, u)
    x.items()
noragen
  • 45
  • 3
3

In neither of these answers the authors seem to understand the concept of updating an object stored in a dictionary nor even of iterating over dictionary items (as opposed to keys). So I had to write one which doesn't make pointless tautological dictionary stores and retrievals. The dicts are assumed to store other dicts or simple types.

def update_nested_dict(d, other):
    for k, v in other.items():
        if isinstance(v, collections.Mapping):
            d_v = d.get(k)
            if isinstance(d_v, collections.Mapping):
                update_nested_dict(d_v, v)
            else:
                d[k] = v.copy()
        else:
            d[k] = v

Or even simpler one working with any type:

def update_nested_dict(d, other):
    for k, v in other.items():
        d_v = d.get(k)
        if isinstance(v, collections.Mapping) and isinstance(d_v, collections.Mapping):
            update_nested_dict(d_v, v)
        else:
            d[k] = deepcopy(v) # or d[k] = v if you know what you're doing
panda-34
  • 4,089
  • 20
  • 25
3

Update to @Alex Martelli's answer to fix a bug in his code to make the solution more robust:

def update_dict(d, u):
    for k, v in u.items():
        if isinstance(v, collections.Mapping):
            default = v.copy()
            default.clear()
            r = update_dict(d.get(k, default), v)
            d[k] = r
        else:
            d[k] = v
    return d

The key is that we often want to create the same type at recursion, so here we use v.copy().clear() but not {}. And this is especially useful if the dict here is of type collections.defaultdict which can have different kinds of default_factorys.

Also notice that the u.iteritems() has been changed to u.items() in Python3.

Daniel
  • 1,783
  • 2
  • 15
  • 25
3
def update(value, nvalue):
    if not isinstance(value, dict) or not isinstance(nvalue, dict):
        return nvalue
    for k, v in nvalue.items():
        value.setdefault(k, dict())
        if isinstance(v, dict):
            v = update(value[k], v)
        value[k] = v
    return value

use dict or collections.Mapping

honmaple
  • 889
  • 1
  • 8
  • 12
3

I recommend to replace {} by type(v)() in order to propagate object type of any dict subclass stored in u but absent from d. For example, this would preserve types such as collections.OrderedDict:

Python 2:

import collections

def update(d, u):
    for k, v in u.iteritems():
        if isinstance(v, collections.Mapping):
            d[k] = update(d.get(k, type(v)()), v)
        else:
            d[k] = v
    return d

Python 3:

import collections.abc

def update(d, u):
    for k, v in u.items():
        if isinstance(v, collections.abc.Mapping):
            d[k] = update(d.get(k, type(v)()), v)
        else:
            d[k] = v
    return d
Nico
  • 31
  • 1
3

Thanks to hobs for his comment on Alex's answer. Indeed update({'k1': 1}, {'k1': {'k2': 2}}) will cause TypeError: 'int' object does not support item assignment.

We should check the types of the input values at the beginning of the function. So, I suggest the following function, which should solve this (and other) problem.

Python 3:

from collections.abc import Mapping


def deep_update(d1, d2):
    if all((isinstance(d, Mapping) for d in (d1, d2))):
        for k, v in d2.items():
            d1[k] = deep_update(d1.get(k), v)
        return d1
    return d2
Sultan
  • 834
  • 1
  • 8
  • 16
2

I know this question is pretty old, but still posting what I do when I have to update a nested dictionary. We can use the fact that dicts are passed by reference in python Assuming that the path of the key is known and is dot separated. Forex if we have a dict named data:

{
"log_config_worker": {
    "version": 1, 
    "root": {
        "handlers": [
            "queue"
        ], 
        "level": "DEBUG"
    }, 
    "disable_existing_loggers": true, 
    "handlers": {
        "queue": {
            "queue": null, 
            "class": "myclass1.QueueHandler"
        }
    }
}, 
"number_of_archived_logs": 15, 
"log_max_size": "300M", 
"cron_job_dir": "/etc/cron.hourly/", 
"logs_dir": "/var/log/patternex/", 
"log_rotate_dir": "/etc/logrotate.d/"
}

And we want to update the queue class, the path of the key would be - log_config_worker.handlers.queue.class

We can use the following function to update the value:

def get_updated_dict(obj, path, value):
    key_list = path.split(".")

    for k in key_list[:-1]:
        obj = obj[k]

    obj[key_list[-1]] = value

get_updated_dict(data, "log_config_worker.handlers.queue.class", "myclass2.QueueHandler")

This would update the dictionary correctly.

dYale
  • 1,541
  • 1
  • 16
  • 19
ipsuri
  • 21
  • 1
2

I made a simple function, in which you give the key, the new value and the dictionary as input, and it recursively updates it with the value:

def update(key,value,dictionary):
    if key in dictionary.keys():
        dictionary[key] = value
        return
    dic_aux = []
    for val_aux in dictionary.values():
        if isinstance(val_aux,dict):
            dic_aux.append(val_aux)
    for i in dic_aux:
        update(key,value,i)
    for [key2,val_aux2] in dictionary.items():
        if isinstance(val_aux2,dict):
            dictionary[key2] = val_aux2

dictionary1={'level1':{'level2':{'levelA':0,'levelB':1}}}
update('levelB',10,dictionary1)
print(dictionary1)

#output: {'level1': {'level2': {'levelA': 0, 'levelB': 10}}}

Hope it answers.

  • So far, your solution has been the only one that's worked for me :) There's 2 issues I have to solve now though. 1 - If there is a key with the same name in a different nested dict, it only updates the first occurrence of the key and 2 - Updating list values – Eitel Dagnin Mar 04 '22 at 10:53
  • Example of list mentioned above: `{"thirdClass": [{"my_string": "my_list","my_int": 3,"my_bool": True}]}` – Eitel Dagnin Mar 04 '22 at 10:55
  • UPDATE: I used this solution and added some additional code for getting dicts from lists (as mentioned above) as well as specifying the name of a nested dict IF there's keys with the same names. Answer posted below! – Eitel Dagnin Mar 07 '22 at 07:30
0

If you want to replace a "full nested dictionary with arrays" you can use this snippet :

It will replace any "old_value" by "new_value". It's roughly doing a depth-first rebuilding of the dictionary. It can even work with List or Str/int given as input parameter of first level.

def update_values_dict(original_dict, future_dict, old_value, new_value):
    # Recursively updates values of a nested dict by performing recursive calls

    if isinstance(original_dict, Dict):
        # It's a dict
        tmp_dict = {}
        for key, value in original_dict.items():
            tmp_dict[key] = update_values_dict(value, future_dict, old_value, new_value)
        return tmp_dict
    elif isinstance(original_dict, List):
        # It's a List
        tmp_list = []
        for i in original_dict:
            tmp_list.append(update_values_dict(i, future_dict, old_value, new_value))
        return tmp_list
    else:
        # It's not a dict, maybe a int, a string, etc.
        return original_dict if original_dict != old_value else new_value
marc_s
  • 732,580
  • 175
  • 1,330
  • 1,459
ZettaCircl
  • 835
  • 10
  • 15
0

Yes! And another solution. My solution differs in the keys that are being checked. In all other solutions we only look at the keys in dict_b. But here we look in the union of both dictionaries.

Do with it as you please

def update_nested(dict_a, dict_b):
    set_keys = set(dict_a.keys()).union(set(dict_b.keys()))
    for k in set_keys:
        v = dict_a.get(k)
        if isinstance(v, dict):
            new_dict = dict_b.get(k, None)
            if new_dict:
                update_nested(v, new_dict)
        else:
            new_value = dict_b.get(k, None)
            if new_value:
                dict_a[k] = new_value
zwep
  • 1,207
  • 12
  • 26
0

Another way of using recursion:

def updateDict(dict1,dict2):
    keys1 = list(dict1.keys())
    keys2= list(dict2.keys())
    keys2 = [x for x in keys2 if x in keys1]
    for x in keys2:
        if (x in keys1) & (type(dict1[x]) is dict) & (type(dict2[x]) is dict):
            updateDict(dict1[x],dict2[x])
        else:
            dict1.update({x:dict2[x]})
    return(dict1)
yusuzech
  • 5,896
  • 1
  • 18
  • 33
0

Credit to: @Gustavo Alves Casqueiro for original answer

I honestly would have preferred using a lib that could do the heavy lifting for me, but I just couldn't find something that did what I needed.

I have only added a couple of additional checks to this function.

I have included a check for lists within a dict and added a parameter for the name of a nested dict to correctly update the nested dict KEY when there may be another KEY within the OUTER dict with the same name.

Updated function:

def update(dictionary: dict[str, any], key: str, value: any, nested_dict_name: str = None) -> dict[str, any]:
    if not nested_dict_name:  # if current (outermost) dict should be updated
        if key in dictionary.keys():  # check if key exists in current dict
            dictionary[key] = value
            return dictionary
    else:  # if nested dict should be updated
        if nested_dict_name in dictionary.keys():  # check if dict is in next layer
            if isinstance(dictionary[nested_dict_name], dict):
                if key in dictionary[nested_dict_name].keys():  # check if key exists in current dict
                    dictionary[nested_dict_name][key] = value
                    return dictionary
            if isinstance(dictionary[nested_dict_name], list):
                list_index = random.choice(range(len(dictionary[nested_dict_name])))  # pick a random dict from the list

                if key in dictionary[nested_dict_name][list_index].keys():  # check if key exists in current dict
                    dictionary[nested_dict_name][list_index][key] = value
                    return dictionary

    dic_aux = []

    # this would only run IF the above if-statement was not able to identity and update a dict
    for val_aux in dictionary.values():
        if isinstance(val_aux, dict):
            dic_aux.append(val_aux)

    # call the update function again for recursion
    for i in dic_aux:
        return update(dictionary=i, key=key, value=value, nested_dict_name=nested_dict_name)

Original dict:

{
    "level1": {
        "level2": {
            "myBool": "Original",
            "myInt": "Original"
        },
        "myInt": "Original",
        "myBool": "Original"
    },
    "myStr": "Original",
    "level3": [
        {
            "myList": "Original",
            "myInt": "Original",
            "myBool": "Original"
        }
    ],
    "level4": [
        {
            "myList": "Original",
            "myInt": "UPDATED",
            "myBool": "Original"
        }
    ],
    "level5": {
        "level6": {
            "myBool": "Original",
            "myInt": "Original"
        },
        "myInt": "Original",
        "myBool": "Original"
    }
}

Data for updating (using pytest):

@pytest.fixture(params=[(None, 'myStr', 'UPDATED'),
                        ('level1', 'myInt', 'UPDATED'),
                        ('level2', 'myBool', 'UPDATED'),
                        ('level3', 'myList', 'UPDATED'),
                        ('level4', 'myInt', 'UPDATED'),
                        ('level5', 'myBool', 'UPDATED')])
def sample_data(request):
    return request.param

The 'UPDATED' parameter doesn't make sense in this smaller use case (since I could just hard-code it), but for simplicity when reading the logs, I didn't want to see multiple data-types and just made it show me an 'UPDATED' string.

Test:

@pytest.mark.usefixtures('sample_data')
def test_this(sample_data):
    nested_dict, param, update_value = sample_data

    if nested_dict is None:
        print(f'\nDict Value: Level0\nParam: {param}\nUpdate Value: {update_value}')
    else:
        print(f'\nDict Value: {nested_dict}\nParam: {param}\nUpdate Value: {update_value}')

    # initialise data dict
    data_object = # insert data here (see example dict above)

    # first print as is
    print(f'\nOriginal Dict:\n{data_object}')

    update(dictionary=data_object,
           key=param,
           value=update_value,
           nested_dict_name=nested_dict)

    # print updated
    print(f'\nUpdated Dict:\n{data_object}')

There is one caveat, when you have a dict like this:

{
    "level1": {
        "level2": {
            "myBool": "Original"
        },
        "myBool": "Original"
    },
    "level3": {
        "level2": {
            "myBool": "Original"
        },
        "myInt": "Original"
    }
}

Where level2 is under level1 AND level3. This would require making using of a list or something with the nested_dict_name and passing in the name of the outer dict AND inner dict (['level5', 'level2']) and then somehow looping through the values to find that dict.

However, since I haven't yet ran into this issue for the data objects I use, I haven't spent the time trying to solve this "issue".

Eitel Dagnin
  • 959
  • 4
  • 24
  • 61
0

Convert your dictionaries into NestedDict

from ndicts.ndicts import NestedDict

dictionary1 = {'level1': {'level2': {'levelA': 0, 'levelB': 1}}}
update = {'level1': {'level2': {'levelB': 10}}}

nd, nd_update = NestedDict(dictionary1), NestedDict(update)

Then just use update

>>> nd.update(nd_update)
>>> nd
NestedDict({'level1': {'level2': {'levelA': 0, 'levelB': 10}}})

If you need the result as a dictionary nd.to_dict()

To install ndicts pip install ndicts

edd313
  • 1,109
  • 7
  • 20
0

d is dict to update, u is dict-updater.

def recursively_update_dict(d: dict, u: dict):
    for k, v in u.items():
        if isinstance(v, dict):
            d.setdefault(k, {})
            recursively_update_dict(d[k], v)
        else:
            d[k] = v

Or for defaultdict

from collections import defaultdict

def recursively_update_defaultdict(d: defaultdict[dict], u: dict):
    for k, v in u.items():
        if isinstance(v, dict):
            recursively_update_dict(d[k], v)
        else:
            d[k] = v
Den Avrondo
  • 89
  • 1
  • 6
0

Basic solution (in case someone arrives here looking for that)

dictionary1 = {
    "level1": {
        "level2": {"levelA": 0, "levelB": 1}
    }
}

dictionary1.update({
    "level1": {
        "level2": {
            **dictionary1["level1"]["level2"],
            "levelB": 10
        }
    }
})

Result:

{'level1': {'level2': {'levelA': 0, 'levelB': 10}}}
Pascal Polleunus
  • 2,411
  • 2
  • 28
  • 30
0

Here's what I use:

from collections.abc import MutableMapping as Map

def merge(d, v):
    """
    Merge two dictionaries.

    Merge dict-like `v` into dict-like `d`. In case keys between them
    are the same, merge their sub-dictionaries. Otherwise, values in  
    `v` overwrite `d`.
    """
    for key in v:
        if key in d and isinstance(d[key], Map) and isinstance(v[key], Map):
            d[key] = merge(d[key], v[key])
        else:
            d[key] = v[key]
    return d
merge(dictionary1, update)
print(dictionary1)

>>> {'level1': {'level2': {'levelA': 0, 'levelB': 10}}}
Hans Bouwmeester
  • 1,121
  • 1
  • 17
  • 19
0

The answer from @kepler helps me, but when I run

from pydantic.utils import deep_update
dict1 = deep_update(dict1, dict2)

a warning comes out:

UserWarning: `pydantic.utils:deep_update` has been removed. We are importing from `pydantic.v1.utils:deep_update` instead.See the migration guide for more details: https://docs.pydantic.dev/latest/migration/
  f'`{import_path}` has been removed. We are importing from `{new_location}` instead.'

So I change the import command to

from pydantic.v1.utils import deep_update
Just
  • 11
  • 1
-1

a new Q how to By a keys chain

dictionary1={'level1':{'level2':{'levelA':0,'levelB':1}},'anotherLevel1':{'anotherLevel2':{'anotherLevelA':0,'anotherLevelB':1}}}
update={'anotherLevel1':{'anotherLevel2':1014}}
dictionary1.update(update)
print dictionary1
{'level1':{'level2':{'levelA':0,'levelB':1}},'anotherLevel1':{'anotherLevel2':1014}}
-1

you could try this, it works with lists and is pure:

def update_keys(newd, dic, mapping):
  def upsingle(d,k,v):
    if k in mapping:
      d[mapping[k]] = v
    else:
      d[k] = v
  for ekey, evalue in dic.items():
    upsingle(newd, ekey, evalue)
    if type(evalue) is dict:
      update_keys(newd, evalue, mapping)
    if type(evalue) is list:
      upsingle(newd, ekey, [update_keys({}, i, mapping) for i in evalue])
  return newd
Craig N.
  • 1
  • 1
  • 4
-4

That's a bit to the side but do you really need nested dictionaries? Depending on the problem, sometimes flat dictionary may suffice... and look good at it:

>>> dict1 = {('level1','level2','levelA'): 0}
>>> dict1['level1','level2','levelB'] = 1
>>> update = {('level1','level2','levelB'): 10}
>>> dict1.update(update)
>>> print dict1
{('level1', 'level2', 'levelB'): 10, ('level1', 'level2', 'levelA'): 0}
Nas Banov
  • 28,347
  • 6
  • 48
  • 67
  • 6
    The nested structure comes from incoming json datasets, so I would like to keep them intact,... – jay_t Jul 13 '10 at 07:57