Python nested dicts using items in a list

Question

This is a question relative to the solution provided here, it involves the following code as solution:

from collections import MutableMapping

def set_value(d, keys, newkey, newvalue, default_factory=dict):
    """
    Equivalent to `reduce(dict.get, keys, d)[newkey] = newvalue`
    if all `keys` exists and corresponding values are of correct type
    """
    for key in keys:
        try:
            val = d[key]
        except KeyError:
            val = d[key] = default_factory()
        else:
            if not isinstance(val, MutableMapping):
                val = d[key] = default_factory()
        d = val
    d[newkey] = newvalue

I'm hoping someone could provide me some explanation why this code works. I'm confused how the passed in dict 'd' doesn't get constantly overwritten where d = val. How does the dict 'd' keep getting further nested dictionaries without ever indexing to the next node? Sorry, if that doesn't make sense, i don't understand how this works.

Thanks for your help!

Martijn Pieters · Answer 1 · 2013-09-08T23:38:32.453

d is rebound; the variable is updated to point to val in each loop.

For each key in keys, either the key is found (val = d[key] succeeds) or the default_factory() is used to create a new value for that key.

If the key was found but the value was not a MutableMapping type, the found value is replaced with a new default_factory() result.

Once the new value has been determined for this level, d is told to forget about the old dictionary and pointed to the new instead.

Rebinding does not change the old value. It merely stops referring to that old value.

Let's use a simple example:

>>> d = {'foo': {}}
>>> keys = ['foo']
>>> newkey = 'bar'
>>> newval = 'eggs'
>>> original = d

At the start, original and d are the same object. Think of names here as paper labels, and their values as balloons. The labels are tied with string to the balloons. In the above example, the d and original labels are both tied to the same dictionary balloon.

When we enter the for key in keys loop, the d[key] lookup succeeds and val is tied to the result of d['foo'], an empty dictionary:

>>> key = keys[0]
>>> key
'foo'
>>> val = d[key]
>>> val
{}

This is a regular Python dictionary, and isinstance(val, MutableMapping) is True. Next line rebinds the d label to that dictionary. The string is simply untied from the original dictionary and now attached to the same balloon val is tied to:

>>> d = val
>>> d
{}
>>> original
{'foo': {}}
>>> d is val
True
>>> d is original
False

The original dictionary was not altered by the rebinding!

Having run out of keys (there was only one in keys), the next part then assigns newval to d[newkey]:

>>> d[newkey] = newval
>>> d
{'bar': 'eggs'}

However, d is not the only label attached to this dictionary balloon. Dictionaries themselves contain keys and values, both of which are labels that are tied to balloons too! The original label is still tied to the outer dictionary balloon, and it has a foo key associated value, which was tied to a nested dictionary, and it is this nested dictionary we just changed:

>>> original
{'foo': {'bar': 'eggs'}}

The algorithm merely followed along labels via strings to new dictionaries.

Using more complex key combinations just means more strings are being followed, with perhaps an extra dictionary being pumped up to be tied in.

score 1 · Accepted Answer · answered Sep 09 '13 at 00:07

I think your question boils down to:

Why does d[newkey] = newvalue modify the object, while d = var does not do anything to the object?

It is just the case that in Python, you can modify a mutable object in a function, but you can't change what object the outer name refers to.

Python nested dicts using items in a list

2 Answers2