Python 3 map dictionary update method to a list of other dictionaries

Question

In Python 2 I can do the following:

>> d = {'a':1}
>> extras = [{'b':2}, {'c':4}]
>> map(d.update, extras)
>> d['c']
>> 4

In Python 3 in get a KeyError:

>> d = {'a':1}
>> extras = [{'b':2}, {'c':4}]
>> map(d.update, extras)
>> d['c']
>> KeyError: 'c'

I would like to achieve the same behavior in Python 3 as in Python 2.

I understand that map in Python 3 will return an iterator (lazy evaluation and whatnot), which has to be iterated for the dictionary to be updated.

I had assumed the d['c'] key lookup would trigger the map iteration somehow, which is not the case.

Is there a pythonic way to achieve this behavior without writing a for loop, which I find to be verbose compared to map.

I have thought of using list comprehensions:

>> d = {'a':1}
>> extras = [{'b':2}, {'c':4}]
>> [x for x in map(d.update, extras)]
>> d['c']
>> 4

But it does not seem pythonic.

`list(map(d.update, extras))` will force the iterator to be consumed — jonrsharpe, May 02 '15 at 09:04
The pythonic way *is* to use a for loop! Using map (or a list comprehension) for its side effects rather than its results is definitely *not* pythonic. — Mark Dickinson, May 02 '15 at 09:13

score 12 · Accepted Answer · answered May 02 '15 at 09:07

12

As you note, map in Python 3 creates an iterator, which doesn't (in and of itself) cause any updates to occur:

>>> d = {'a': 1}
>>> extras = [{'b':2}, {'c':4}]
>>> map(d.update, extras)
<map object at 0x105d73c18>
>>> d
{'a': 1}

To force the map to be fully evaluated, you could pass it to list explicitly:

>>> list(map(d.update, extras))
[None, None]
>>> d
{'a': 1, 'b': 2, 'c': 4}

However, as the relevant section of What's new in Python 3 puts it:

Particularly tricky is map() invoked for the side effects of the function; the correct transformation is to use a regular for loop (since creating a list would just be wasteful).

In your case, this would look like:

for extra in extras:
    d.update(extra)

which doesn't result in an unnecessary list of None.

answered May 02 '15 at 09:07

jonrsharpe

115,751
26
228
437

that for loop was just going bite me! thanks anyway – mattgathu May 02 '15 at 09:25
2

Note: instead of `list` one can use `collections.deque(iterable, maxlen=0)` to consume the `iterable` without consuming memory. This also executes only one bytecode so should be slightly more time efficient than an explicit `for` at least in CPython. – Bakuriu May 02 '15 at 11:24
@Bakuriu interesting, I wouldn't have thought of doing that; I didn't know a `deque` *could* have a zero maximum length! – jonrsharpe May 02 '15 at 11:27
brilliant explanation – imak Apr 10 '20 at 21:02

score 1 · Answer 2 · edited May 23 '17 at 11:50

Alongside the @jonrsharpe's explanation that explains the problem clearly In Python 3 you can use collections.ChainMap for such tasks:

>>> from collections import ChainMap
>>> chain=ChainMap(d, *extras)
>>> chain
ChainMap({'a': 1}, {'b': 2}, {'c': 4})
>>> chain['c']
4

But note that if there are duplicate keys, the values from the first mapping get used.

Read more about the advantages of using ChainMap :What is the purpose of collections.ChainMap?

Python 3 map dictionary update method to a list of other dictionaries

2 Answers2

Linked