Merge dictionaries with sub dictionaries

Question

I would like to deep-merge 2 dictionaries which both have at some level one or more dictionaries values with the same key. How can I merge also those internal dictionaries simply? Something like the "**" notation but a deep merge.

Example:

d1 = {"v1": "value1", "sub": {"sv1": "sub value 1"}}
d2 = {"v2": "value2", "sub": {"sv2": "sub value 2"}}
d3 = {**d1, **d2}
print(d3)

Actual result:

{'v1': 'value1', 'sub': {'sv2': 'sub value 2'}, 'v2': 'value2'}

Desired result:

{'v1': 'value1',
 'sub': {'sv1': 'sub value 1', 'sv2': 'sub value 2'},
 'v2': 'value2'}

Noticed the sub -> sv1 -> sub value 1

This is a simple example while the real case have several more levels of dictionaries. Moreover the solution needs to be generic since the structure of the dictionaries is not known in advanced.

Close as duplicate of [this](https://stackoverflow.com/questions/7204805/dictionaries-of-dictionaries-merge)? — Antti Haapala -- Слава Україні, Aug 31 '17 at 06:57
I've been also writing a library that can do a generic 3-way merge on JSON objects (this is even more complicated though). — Antti Haapala -- Слава Україні, Aug 31 '17 at 06:58
@AnttiHaapala it is not a dup, he doesn't considered the cases where both sub-dict have the same keys. He just talking about a deep copying. — Roee Gavirel, Aug 31 '17 at 07:01
"where both sub-dict have the same keys." what do you mean? Your subdicts have *different* keys, `sv1` and `sv2`; the accepted answer **also** handles same keys, i.e. conflict of `sv1` and `sv1` with different values... — Antti Haapala -- Слава Україні, Aug 31 '17 at 07:03
The example is just a light example. I want to handle also a case of `s1->s2->s3->value1` merge with `s1->s2->s3->value2` etc. — Roee Gavirel, Aug 31 '17 at 07:12
And you didn't mention it in **your** question, but instead ask for "something else" — Antti Haapala -- Слава Україні, Aug 31 '17 at 07:18
@AnttiHaapala - Point taken, I edited my question to make it more understandable. — Roee Gavirel, Aug 31 '17 at 07:26

score 2 · Accepted Answer · answered Sep 03 '17 at 06:46

I ended up writing this:

import collections
import copy

def deep_dict_merge(dct1, dct2, override=True) -> dict:
    """
    :param dct1: First dict to merge
    :param dct2: Second dict to merge
    :param override: if same key exists in both dictionaries, should override? otherwise ignore. (default=True)
    :return: The merge dictionary
    """
    merged = copy.deepcopy(dct1)
    for k, v2 in dct2.items():
        if k in merged:
            v1 = merged[k]
            if isinstance(v1, dict) and isinstance(v2, collections.Mapping):
                merged[k] = deep_dict_merge(v1, v2, override)
            elif isinstance(v1, list) and isinstance(v2, list):
                merged[k] = v1 + v2
            else:
                if override:
                    merged[k] = copy.deepcopy(v2)
        else:
            merged[k] = copy.deepcopy(v2)
    return merged

score 1 · Answer 2 · answered Aug 31 '17 at 06:40

1

You already had the first three steps to create d3 with all the common keys:

>>> d1 = {"v1": "value1", "sub": {"sv1": "sub value 1"}}
>>> d2 = {"v2": "value2", "sub": {"sv2": "sub value 2"}}
>>> d3 = {**d1, **d2}  # initial merge of common keys

There just needs to be a follow-on step to merge in all the "sub" entries:

>>> for d in (d1, d2): # merge the "sub" entries
        d3['sub'].update(d['sub'])

That gives the desired result:

>>> d3
{'v1': 'value1', 'sub': {'sv2': 'sub value 2', 'sv1': 'sub value 1'}, 'v2': 'value2'}

Hope this helps :-)

answered Aug 31 '17 at 06:40

Raymond Hettinger

216,523
63
388
485

`d2['sub']` ending up having the modifications might be an unwanted effect. – Ilja Everilä Aug 31 '17 at 06:46
Thanks, that is a good approach. but I'm looking for a more generic case. The example I gave here was just to make a point. In my real case I don't know the keys and moreover it may have several levels not just two. – Roee Gavirel Aug 31 '17 at 06:48
@RoeeGavirel I gave the answer to the question as asked and editted multiple times. It sounds like your actual problem is underspecified. – Raymond Hettinger Aug 31 '17 at 06:54

Nirmi · Answer 3 · 2017-09-04T05:26:30.153

1

for python3:

d1 = {"v1": "value1", "sub": {"sv1": "sub value 1",'newsub':"value 2",'newsubdict':{'ok':'pssible','not_again':{'key1':'just checking'}}}}
d2 = {"v2": "value2", "sub": {"sv2": "sub value 2",'newsubdict':{'fine':'i am working','not_again':{'key2':'Am i right?'}}}}

from copy import deepcopy
d3 = deepcopy(d1)

def combine_key(d2,d3):
  for key in d2.keys():
      if(key in d3):
              d3[key] =d2[key]
      elif(isinstance(d3[key],dict)):
              combine_key(d2[key],d3[key])
      else:
              d3[key].update(d2[key])

combine_key(d2,d3)
print(d3) #{'v1': 'value1', 'sub': {'sv1': 'sub value 1', 'newsub': 'value 2', 'newsubdict': {'ok': 'pssible', 'not_again': {'key1': 'just checking', 'key2': 'Am i right?'}, 'fine': 'i am working'}, 'sv2': 'sub value 2'}, 'v2': 'value2'}

edited Sep 04 '17 at 05:26

answered Aug 31 '17 at 07:30

Nirmi

356
3
11

Thanks, but this is a specific answer, I'm looking for a more generic one. – Roee Gavirel Aug 31 '17 at 09:23
Generic mean you have no idea about D1 and D2 dictionary structure. So it can be number of nested dictionary with same key is it? – Nirmi Aug 31 '17 at 12:04
Yeap, that is exactly the situation I'm talking about. – Roee Gavirel Aug 31 '17 at 12:58
@RoeeGavirel You can take a look at updated code now.I have modified dictionary for 3 to 4 nested dictionary. – Nirmi Aug 31 '17 at 16:23
This is a good approach, close to what I ended writing. But I think the use of the "no" is wrong, what if one actually have this key. Better use `if key in d3` – Roee Gavirel Sep 03 '17 at 06:48

Merge dictionaries with sub dictionaries

3 Answers3