38

Is there a library in Python that I can use to deep merge dictionaries:

The following:

a = { 'first' : { 'all_rows' : { 'pass' : 'dog', 'number' : '1' } } }
b = { 'first' : { 'all_rows' : { 'fail' : 'cat', 'number' : '5' } } }

When i combine I want this to look like:

a = { 'first' : { 'all_rows' : { 'pass' : 'dog', 'fail' : 'cat', 'number' : '5' } } }
evolution
  • 4,332
  • 5
  • 29
  • 34

1 Answers1

73

I hope I don't reinvent the wheel but the solution is fairly short. And, superfun to code.

def merge(source, destination):
    """
    run me with nosetests --with-doctest file.py

    >>> a = { 'first' : { 'all_rows' : { 'pass' : 'dog', 'number' : '1' } } }
    >>> b = { 'first' : { 'all_rows' : { 'fail' : 'cat', 'number' : '5' } } }
    >>> merge(b, a) == { 'first' : { 'all_rows' : { 'pass' : 'dog', 'fail' : 'cat', 'number' : '5' } } }
    True
    """
    for key, value in source.items():
        if isinstance(value, dict):
            # get node or create one
            node = destination.setdefault(key, {})
            merge(value, node)
        else:
            destination[key] = value

    return destination

So the idea is to copy the source to the destination, and every time it's a dict in the source, recurse. So indeed you will have a bug if in A a given element contains a dict and in B any other type.

[EDIT] as said in comments the solution was already here : https://stackoverflow.com/a/7205107/34871

Community
  • 1
  • 1
vincent
  • 6,368
  • 3
  • 25
  • 23
  • This is excellent and yes - fun. Should have taken a closer look at the DUP earlier. Nice use of recursion. From brief testing, it seems fit for purpose. The data i'm working with is always of type dict in a or b. – evolution Dec 18 '13 at 20:01
  • 2
    +1 - This seems a lot more straight forward that the DUP one in the case that we don't care about conflicts – urban Sep 20 '17 at 08:52
  • 8
    Just beware that this is modifying the `destination` dict *in place!* – Arel Mar 07 '19 at 22:20
  • 1
    What @Arel said. It would probably make sense to remove the return statement, as it's a bit deceptive. – naught101 Apr 04 '19 at 23:45
  • 2
    Fails with case: ({'b': {'c': 1}}, {'b': 1}), as {'b': {'c': 1}} is expected. Fix is to check for primitive types after setdefault, and default to {} if so. – onesiumus Jul 18 '19 at 08:01
  • 1
    The code does not produce the result documented in the comment. The result has 'number':'1' not 'number':'5'. Either the comment or the code needs to be fixed. – alecswan Dec 18 '19 at 23:36
  • this library offers several different policies of deep merging dict https://y.tsutsumi.io/2017/04/24/deepmerge-deep-merge-dictionaries-lists-and-more-in-python/ – suzukimilanpaak May 28 '20 at 09:45
  • isn't it better if lists would be joined together instead of being replaced? – ahmadali shafiee Jan 22 '21 at 21:23