Dict merge in a dict comprehension

Question

In python 3.5, we can merge dicts by using double-splat unpacking

>>> d1 = {1: 'one', 2: 'two'}
>>> d2 = {3: 'three'}
>>> {**d1, **d2}
{1: 'one', 2: 'two', 3: 'three'}

Cool. It doesn't seem to generalise to dynamic use cases, though:

>>> ds = [d1, d2]
>>> {**d for d in ds}
SyntaxError: dict unpacking cannot be used in dict comprehension

Instead we have to do reduce(lambda x,y: {**x, **y}, ds, {}), which seems a lot uglier. Why the "one obvious way to do it" is not allowed by the parser, when there doesn't seem to be any ambiguity in that expression?

You can't do that in any other *- or **-unpacking context either. I.e., you can't do `some_function(*x for x in list_lists)`. The unpacking stars aren't real operators and can't occur in expressions. — BrenBarn, Jun 02 '16 at 06:49
`{k: v for d in [d1, d2] for k, v in d.items()}` would be an alternative to your `reduce()`, though "uglier" still. — Ilja Everilä, Jun 02 '16 at 07:07
I believe yet another alternative is `dict(ChainMap(d2, d1))` which I personally don't like because who the hell knows what a `ChainMap` is anyway? — machine yearning, Jun 02 '16 at 07:12
Actually, just `ChainMap(*ds)` seems to be good enough on its own! Nice, you should add that as an answer. — wim, Jun 02 '16 at 07:15
@wim you could edit the question to state that `ChainMap` is acceptable, saying "I want a view of merged dictionaries", "currently I am merging the dictionaries to create a new one" — Antti Haapala -- Слава Україні, Jun 02 '16 at 07:45
also, you need to reverse the arguments to `ChainMap` (first found is used) — Antti Haapala -- Слава Україні, Jun 02 '16 at 07:46
@AnttiHaapala Good point, I'll make that clear in my answer. — machine yearning, Jun 02 '16 at 08:12

score 38 · Accepted Answer · edited Aug 27 '21 at 17:06

It's not exactly an answer to your question but I'd consider using ChainMap to be an idiomatic and elegant way to do what you propose (merging dictionaries in-line):

>>> from collections import ChainMap
>>> d1 = {1: 'one', 2: 'two'}
>>> d2 = {3: 'three'}
>>> ds = [d1, d2]
>>> dict(ChainMap(*ds))
{1: 'one', 2: 'two', 3: 'three'}

Although it's not a particularly transparent solution, since many programmers might not know exactly how a ChainMap works. Note that (as @AnttiHaapala points out) "first found is used" so, depending on your intentions you might need to make a call to reversed before passing your dicts into ChainMap.

>>> d2 = {3: 'three', 2: 'LOL'}
>>> ds = [d1, d2]
>>> dict(ChainMap(*ds))
{1: 'one', 2: 'two', 3: 'three'}

>>> dict(ChainMap(*reversed(ds)))
{1: 'one', 2: 'LOL', 3: 'three'}

score 19 · Answer 2 · answered Jun 02 '16 at 06:57

19

To me, the obvious way is:

d_out = {}
for d in ds:
    d_out.update(d)

This is quick and probably quite performant. I don't know that I can speak for the python developers, but I don't know that your expected version is more easy to read. For example, your comprehension looks more like a set-comprehension to me due to the lack of a :. FWIW, I don't think there is any technical reason (e.g. parser ambiguity) that they couldn't add that form of comprehension unpacking.

Apparently, these forms were proposed, but didn't have universal enough support to warrant implementing them (yet).

answered Jun 02 '16 at 06:57

mgilson

300,191
65
633
696

1

By making it an expression instead of a statement, the possible use-cases are increased for functional styles. If the for loop was always better, there would be no reason to add the merging expression at all - my question is more about why it was intentionally limited to a known predetermined number of operands? – wim Jun 02 '16 at 07:04
@wim -- If you really want a to do this in a functional approach, just wrap this in a function :-). As discussed in the relevant PEP, the reason that it was intentionally limited was because there wasn't strong enough consensus in the community about what syntax to use. Perhaps it will be revisited at some time in the future, but for now, it was omitted so that the parts that everyone did agree on could be scheduled for implemenation. – mgilson Jun 02 '16 at 07:09
1

Funnily enough the comprehension syntax was [part of the implementation](https://www.python.org/dev/peps/pep-0448/#implementation) and got actively removed. – norok2 Jun 12 '19 at 16:56
@norok2 I still can not see any convincing reason for why it was removed (?) – wim Jan 22 '20 at 16:54

Lars Blumberg · Answer 3 · 2023-02-14T08:36:57.587

3

Idiomatic, without ChainMap:

>>> d1 = {1: 'one', 2: 'two'}
>>> d2 = {3: 'three'}
>>> {k: v for d in [d1, d2] for k, v in d.items()}
{1: 'one', 2: 'two', 3: 'three'}

edited Feb 14 '23 at 08:36

answered Nov 13 '22 at 23:19

Lars Blumberg

19,326
11
90
127

score 2 · Answer 4 · edited Jan 09 '22 at 22:48

2

You could use itertools.chain or itertools.chain.from_iterable:

import itertools

ds = [{'a': 1, 'b': 2}, {'c': 30, 'b': 40}]

merged_d = dict(itertools.chain(*(d.items() for d in ds)))
print(merged_d)  # {'a': 1, 'b': 40, 'c': 30}

edited Jan 09 '22 at 22:48

Asclepius

57,944
17
167
143

answered Oct 26 '20 at 12:59

Conchylicultor

4,631
2
37
40

And how do I unpack it in the same line (before attributing to merged_d) – Iuri Guilherme Oct 15 '22 at 08:41

score 1 · Answer 5 · answered Jan 14 '22 at 14:20

Based on this solution and also mentioned by @ilgia-everilä, but making it Py2 compatible and still avoiding intermediate structures. Encapsulating it inside a function makes its use quite readable.

def merge_dicts(*dicts, **extra):
    """
    >>> merge_dicts(dict(a=1, b=1), dict(b=2, c=2), dict(c=3, d=3), d=4, e=4)
    {'a': 1, 'b': 2, 'c': 3, 'd': 4, 'e': 4}
    """
    return dict((
        (k,v)
        for d in dicts
        for k,v in d.items()
    ), **extra)

score 0 · Answer 6 · answered Jan 22 '20 at 15:24

You could define this function:

from collections import ChainMap
def mergeDicts(l):
    return dict(ChainMap(*reversed(list(l))))

You can then use it like this:

>>> d1 = {1: 'one', 2: 'two'}
>>> d2 = {3: 'three'}
>>> ds = [d1, d2]
>>> mergeDicts(ds)
{1: 'one', 2: 'two', 3: 'three'}

Dict merge in a dict comprehension

6 Answers6

Linked

Related