Merge two dictionaries with nested dictionaries into new one, summing same keys and keeping unaltered ones in Python

Question

I have two dictionaries, both with a nested dictionary, and I want to merge them in a bigger dictionary. Some of the keys are different in the two dictionaries, some of them are the same. The ones that are different, I simply want to copy to the new bigger dictionary. The ones that are the same, I want to keep the key both from the dictionary and the nested one the same, and add the values from the nested dictionary.

Sample data:

dict1 = {'C-STD-B&M-L':
              {datetime.date(2015, 4, 30): 0.06875104,
               datetime.date(2015, 5, 1): 0.07842368,
               datetime.date(2015, 5, 2): 0.08919679999999999,
               datetime.date(2015, 5, 16): 0.40798848},
         'G-CAM-MAS-XXS': 
              {datetime.date(2015, 4, 30): 0.0008190299999999999,
               datetime.date(2015, 5, 1): 0.00093426,
               datetime.date(2015, 5, 2): 0.0010626}

dict2 = {'C-STD-B&M-L':
              {datetime.date(2015, 5, 16): 0.075968,
               datetime.date(2015, 5, 17): 0.086656},
         'H-QLD-BAC-STD': 
              {datetime.date(2015, 5, 16): 0.17804999999999999,
               datetime.date(2015, 5, 17): 0.2031,
               datetime.date(2015, 5, 18): 0.23099999999999998}

Expected output:

 new_dict =  
        {'C-STD-B&M-L':
              {datetime.date(2015, 4, 30): 0.06875104,
               datetime.date(2015, 5, 1): 0.07842368,
               datetime.date(2015, 5, 2): 0.08919679999999999,
               datetime.date(2015, 5, 16): 0.40798848 + 0.075968
               datetime.date(2015, 5, 17): 0.086656},
         'G-CAM-MAS-XXS': 
              {datetime.date(2015, 4, 30): 0.0008190299999999999,
               datetime.date(2015, 5, 1): 0.00093426,
               datetime.date(2015, 5, 2): 0.0010626
         'H-QLD-BAC-STD': 
              {datetime.date(2015, 5, 16): 0.17804999999999999,
               datetime.date(2015, 5, 17): 0.2031,
               datetime.date(2015, 5, 18): 0.23099999999999998}

I posted the value as a sum (0.40798848 + 0.075968) just to clarify, I actually need the sum (0.48395648).

I tried to do some for loops to add them but they got really messy, I'll post just for the sake of it.

Sample code:

all_unis_dict = {}
for skus1, subdict1 in unsw_mid_year_dict.items():
    for skus2, subdict2 in unsw_mid_year_dict.items():
        for dates1, days1 in subdict1.items():
            for dates2, days2 in subdict2.items():
                pprint(days1 + days2)

Anyone know a elegant way to do it?

score 1 · Accepted Answer · answered Jun 24 '15 at 05:21

Seems like a use-case for a defaultdict of a defaultdict of int ...

The basic idea is that when you come across a key that isn't in the top-level defaultdict, you add the key (with an associated defaultdict(int) to store the date -> integer map as the value). When you come across a date which isn't in the nested dict, the defaultdict(int) will add the date with a default value of 0 (since int() called with no arguments returns 0).

Here's some code to make it concrete:

from collections import defaultdict
output = defaultdict(lambda: defaultdict(int))
for d in (dict1, dict2):
    for key, values_dict in d.items():
        for date, integer in values_dict.items():
            output[key][date] += integer

If you really want to be thorough, after the fact you can set the default_factory to None to prevent any more "default" behavior from the defaultdict:

output.default_factory = None
for default_date_dict in output.values():
    default_date_dict.default_factory = None

By the way, is there any where I could find more information about this? Having some trouble wrapping my head around it hehe. — Filipe Barreto Peixoto Teles, Jun 24 '15 at 07:05
@iamfbpt -- I think the first thing to do is to start experimenting with `defaultdict` a bit more. The docs are at https://docs.python.org/2/library/collections.html#collections.defaultdict -- A great example to play around with (and explanation) is at http://stackoverflow.com/q/5900578/748858 ... Once you've grokked the basics of defaultdict, the nested one isn't *too* much harder to grasp. — mgilson, Jun 24 '15 at 15:39

Merge two dictionaries with nested dictionaries into new one, summing same keys and keeping unaltered ones in Python

1 Answers1

Linked