Fastest way to merge two dictionaries

Question

In looking at the methods for merging a dictionary in python, I have found two different methods that appear to have the same functionality.

{**dict1, **dict2}

and

dict1 | dict2

Aside from the latter only being available in 3.9 and above, it seems as though these two methods have the same functionality. That is, they both take in two dictionaries like the following:

dict1 = {'a': 1, 'b': 2}
dict2 = {'c': 3, 'd': 4}

and return {'a': 1, 'b': 2, 'c': 3, 'd': 4}

My question is, is one of these methods dramatically (or even slightly) quicker than the other? Are there any other differences in these two methods?

EDIT Thanks for all the replies. In general, it seems as though the two methods listed above aren't significantly different in speed.

One note that was pointed out in the comments is that the ChainMap() function in the collections library does show some improvements when dealing with dictionaries of sufficient scale:

python3 -m timeit -n 1000 "x={'x_'+str(i):True for i in range(10000)};y={'y_'+str(i):True for i in range(10000)};x|y"

1000 loops, best of 5: 7.22 msec per loop

python3 -m timeit -n 1000 "from collections import ChainMap;x={'x_'+str(i):True for i in range(10000)};y={'y_'+str(i):True for i in range(10000)};ChainMap(x,y)"

1000 loops, best of 5: 4.16 msec per loop

There is also `dict1.update(dict2)` if you don't mind mutating `dict1`. — Steven Rumbalski, Sep 28 '22 at 19:49
Let's say they have intersecting keys? How do you want to handle merging them? Replace the values in dict1 with the values in dict2? Put both values into a list/tuple? — Chris, Sep 28 '22 at 19:51
Some one has timed this [#Day27 – Fastest Way to Combine Dictionaries](https://www.realpythonproject.com/day27-fastest-way-to-combine-dictionaries/) using six different methods. That said, just write what's clearest. If you have a speed issue, profile your code and optimize the slow parts. Premature optimization leads to less clear, less maintainable code. — Steven Rumbalski, Sep 28 '22 at 19:56
There are no differences that I'm aware of, and neither is significantly faster. I'm a little confused why a dict union operator was put into the language in the first place, this just seems to add another way to do the same thing. ¯\\_(ツ)_/¯ — wim, Sep 28 '22 at 19:56
@wim The rationale is spelled out in [PEP 584 – Add Union Operators To dict](https://peps.python.org/pep-0584/#motivation) — Steven Rumbalski, Sep 28 '22 at 19:59
@StevenRumbalski Yes, I'm aware, but I find the PEP unconvincing. For example it says _`{**d1, **d2}` ignores the types of the mappings and always returns a dict_ - well, so does the union operator. It calls the unpacking "ugly" but that seems opinion-based, I find it more clear and obvious personally. Regarding `|=`, it says _`e.update(d2)` is not an expression and needs a temporary variable_ - well, so what? — wim, Sep 28 '22 at 20:01

gog · Answer 1 · 2022-09-28T20:14:32.857

-1

The key difference is that ** supports any mapping, while | requires both operands to be dicts:

class C:
    def keys(self):
        return 'xyz'
    def __getitem__(self, item):
        return item + '!'

a = {'foo': 'bar'}
c = C()

print( {**a, **c} ) # ok
print( a | c )      # nope

Note that unlike |, |= does work for mappings, so a |= c would be fine. This is similar to list+list vs list += iterable.

edited Sep 28 '22 at 20:14

answered Sep 28 '22 at 19:59

gog

10,367
2
24
38

`|` doesn't require that. But you're obliged to implement `C.__ror__` for defining what you want to happen here, explicitly. – wim Sep 28 '22 at 20:07
And a related difference is that `{**a, **b}` requires the *result* to be a dict, whereas `a | b` is an `OrderedDict` for `OrderedDict` `a`, for example. – Ry- Sep 28 '22 at 20:10
@wim: in that sense, you can say every operator is supported for everything.... if you implement it. The _built-in_ `|` only supports `dict|dict`. – gog Sep 28 '22 at 20:12
@Ry: this is because you can't redefine literals. `{ whatever }` is always a dict (well, when it's not a set ;) – gog Sep 28 '22 at 20:13
@gog `|` is not a built-in and doesn't "support" anything. It's the `dict` type which supports operator `|`, by implementing datamodel hook `__or__`. I still think this claim in answer _"`|` requires both operands to be dicts"_ is wrong or at best poorly worded. – wim Sep 28 '22 at 20:31
@gog: I literally just gave an example where a built-in mapping type other than dict supports `|`. – Ry- Sep 28 '22 at 21:36
@Ry: `OrderedDict` _is_ `dict` and not just some mapping. The point being, the `|` implementation is nominal, not structural. Check out https://github.com/python/cpython/blob/main/Objects/dictobject.c#L3570 – gog Sep 28 '22 at 22:30

Fastest way to merge two dictionaries

1 Answers1