How can I combine dictionaries with the same keys?

Question

I have a list of dictionaries like so:

dicts = [
    {'key_a': valuex1,
     'key_b': valuex2,
     'key_c': valuex3},

    {'key_a': valuey1,
     'key_b': valuey2,
     'key_c': valuey3},

    {'key_a': valuez1,
     'key_b': valuez2,
     'key_c': valuez3}
]

I would like to take these and construct a big dictionary like so:

big_dict = {
    'key_a': [valuex1, valuey1, valuez1],
    'key_b': [valuex2, valuey2, valuez2],
    'key_c': [valuex3, valuey3, valuez3]
}

Is there any elegant "zip"-like way for me to do this?

All the keys are always going to be identical.

score 16 · Accepted Answer · edited Dec 10 '19 at 03:21

16

big_dict = {}
for k in dicts[0]:
    big_dict[k] = [d[k] for d in dicts]

Or, with a dict comprehension:

{k: [d[k] for d in dicts] for k in dicts[0]}

edited Dec 10 '19 at 03:21

Boris Verkhovskiy

14,854
11
100
103

answered Jul 18 '12 at 01:58

Ned Batchelder

364,293
75
561
662

2

Just adding a detail to avoid an KeyError when any of the sub-dictionaries doesn't have all keys: {k: [d[k] for d in results if k in d.keys()] for k in results[0]} – Ricardo Guerreiro Sep 28 '21 at 09:18

score 5 · Answer 2 · edited Dec 10 '19 at 03:25

You can use collections.defaultdict. The benefit of this solution is it does not require keys to be consistent across dictionaries, and it still maintains the minimum O(n) time complexity.

from collections import defaultdict

dict_list = [{'key_a': 'valuex1', 'key_b': 'valuex2', 'key_c': 'valuex3'},
             {'key_a': 'valuey1', 'key_b': 'valuey2', 'key_c': 'valuey3'},
             {'key_a': 'valuez1', 'key_b': 'valuez2', 'key_c': 'valuez3'}]            

d = defaultdict(list)
for myd in dict_list:
    for k, v in myd.items():
        d[k].append(v)

Result:

print(d)

defaultdict(list,
            {'key_a': ['valuex1', 'valuey1', 'valuez1'],
             'key_b': ['valuex2', 'valuey2', 'valuez2'],
             'key_c': ['valuex3', 'valuey3', 'valuez3']})

score 2 · Answer 3 · answered Jul 18 '12 at 01:58

2

If all the dicts have the same set of keys, this will work:

dict((k, [d[k] for d in dictList]) for k in dictList[0])

If they may have different keys, you'll need to first built a set of keys by doing set unions on the keys of the various dicts:

allKeys = reduce(operator.or_, (set(d.keys()) for d in dictList), set())

Then you'll need to protect against missing keys in some dicts:

dict((k, [d[k] for d in [a, b] if k in d]) for k in allKeys)

answered Jul 18 '12 at 01:58

BrenBarn

242,874
37
412
384

4

You could use a dictionary comprehension: `{k: [d[k] for d in dictList[ for k in dictList[0]}` – Blender Jul 18 '12 at 01:59
Yes, if you're using a Python version that supports dictionary comprehensions. I'm using 2.6 myself so I don't have that. – BrenBarn Jul 18 '12 at 02:02
a dict comprehension is syntactically synonymous to a generator expression where `{key: value ...}` is replaced with `dict((key, value) ...)`, which is even valid in 2.6 – Joel Cornett Jul 18 '12 at 02:30

score 0 · Answer 4 · answered Sep 12 '18 at 17:43

If you are happy to use a 3rd party library you can use Pandas. The pd.DataFrame constructor accepts a list of dictionaries directly:

import pandas as pd

res = pd.DataFrame(dictList).to_dict(orient='list')

{'key_a': ['valuex1', 'valuey1', 'valuez1'],
 'key_b': ['valuex2', 'valuey2', 'valuez2'],
 'key_c': ['valuex3', 'valuey3', 'valuez3']}

score -1 · Answer 5 · answered Dec 03 '14 at 18:39

You can merge dictionaries in the following way:

def merge_dicts(dict_list, separator=''):
    """
    Merges list of dictionaries to a single dictionary, Concatenates values with the same key.
    :param dict_list: list of dictionaries to be merged.
    :param separator: separator to be inserted between values of same key.
    :return: Merged dictionary.
    """
    return {k1: separator.join([d[k1] for d in dict_list if k1 in d])
            for k1 in set(reduce(lambda x, y: x+y, [k.keys() for k in dict_list]))
    }

How can I combine dictionaries with the same keys?

5 Answers5

Linked

Related