10

I feel like I saw a way to do this recently. Say I've got an empty dict and I want to set a value in a nested dict inside that empty dict, but obviously that nested dict hasn't been created yet. Is there a 1-line way to create the intermediate keys? This is what I want to do:

mydict = {}
mydict['foo']['bar']['foobar'] = 25

If you execute this code you'll get a KeyError exception for 'foo'. Is there a function to create intermediate keys?

Thanks.

skandocious
  • 814
  • 9
  • 21
  • possible duplicate of [Generating python dict keys on the fly](http://stackoverflow.com/questions/3405073/generating-python-dict-keys-on-the-fly) – unutbu Apr 18 '12 at 21:47

3 Answers3

19
from collections import defaultdict
recursivedict = lambda: defaultdict(recursivedict)
mydict = recursivedict()

When you access mydict['foo'], it sets mydict['foo'] to another recursivedict. It'll actually construct a recursivedict for mydict['foo']['bar']['foobar'] as well, but then it'll get thrown out by assigning that to 25.

Danica
  • 28,423
  • 6
  • 90
  • 122
  • This is exactly what I had remembered reading about-- thanks. – skandocious Apr 18 '12 at 21:47
  • This might break on this `mydict['foo'] = 15⏎ mydict['foo']['bar']['foobar'] = 25`. On a large codebase the OP can't remember what value he assigned earlier. – nehem Nov 12 '15 at 09:57
  • @itsneo That's true. I don't think there's a good way around that; you'd have to make `__getitem__` return wrappers around the element stored overriding that element's `__getitem__`, which would make it tricky to put a list or whatever into the dict. – Danica Nov 12 '15 at 19:31
  • Awesome! Thanks, that is the droid I was looking for! – darkless Jan 07 '16 at 16:02
5

An alternative option - depending on your uses, is to use tuples as keys instead of nested dictionaries:

mydict = {}
mydict['foo', 'bar', 'foobar'] = 25

This will work perfectly well unless you want to get a branch of the tree at any point (you can't get mydict['foo'] in this case).

If you knew how many layers of nesting you want, you could also use functools.partial instead of lambda.

from functools import partial
from collections import defaultdict

tripledict = partial(defaultdict, partial(defaultdict, dict))
mydict = tripledict()
mydict['foo']['bar']['foobar'] = 25

Which some people find more readable, and is faster to create instances of than the equivalent lambda-based solution:

python -m timeit -s "from functools import partial" -s "from collections import defaultdict" -s "tripledefaultdict = partial(defaultdict, partial(defaultdict, dict))" "tripledefaultdict()"
1000000 loops, best of 3: 0.281 usec per loop

python -m timeit -s "from collections import defaultdict" -s "recursivedict = lambda: defaultdict(recursivedict)" "recursivedict()"
1000000 loops, best of 3: 0.446 usec per loop

Although, as always, there is no point optimising until you know there is a bottleneck, so pick what is most useful and readable before what is fastest.

Gareth Latty
  • 86,389
  • 17
  • 178
  • 183
  • Interesting stuff. I never thought to use a tuple as a dict key. Though this doesn't help in my case it's certainly useful-- thanks! – skandocious Apr 18 '12 at 22:17
  • Of course, it's possible to get tree branches with the tuple-based solution through something like `prefix = ('foo', 'bar'); l = len(prefix); branch = { k[l:]: v for k, v in mydict.items() if k[:l] == prefix}` -- though that needs to iterate over all the keys. – Danica Apr 18 '12 at 22:19
  • @Dougal Yeah, at that point, you are better off just using the nested ``dict``s. – Gareth Latty Apr 18 '12 at 22:19
  • @Lattyware Eh, depends. Normally yes, but if you need to do a ton of deep element accesses and only very rarely need to make branches, it might be faster to use the tuple approach (because that only does one dict hashing per access). I have the feeling that in the OP's case this isn't really a consideration, though. – Danica Apr 18 '12 at 22:21
  • @Dougal I actually saw [a question](stackoverflow.com/questions/10182841/two-dimensional-vs-one-dimensional-dictionary-efficiency-in-python) recently about that which surprised me, it seems nested dicts are actually faster for access than tuples. – Gareth Latty Apr 18 '12 at 22:29
  • @Lattyware Whoa! That is surprising. Your comment about the fast path for string dictionaries is probably most of the reason, but I wonder if there aren't also more hash collisions in the tuple case. – Danica Apr 18 '12 at 22:36
  • @Dougal Yeah, I'm not too sure as to why it is. Obviously that's only a small test, so take it with a grain of salt, but still worth thinking about. – Gareth Latty Apr 18 '12 at 22:38
0

Not sure why you'd want to, but:

>>> from collections import defaultdict as dd
>>> mydict = dd(lambda: dd(lambda: {}))
>>> mydict['foo']['bar']['foobar'] = 25
>>> mydict
defaultdict(<function <lambda> at 0x021B8978>, {'foo': defaultdict(<function <lambda> at 0x021B8618>, {'bar': {'foobar': 25}})})
MattH
  • 37,273
  • 11
  • 82
  • 84
  • This only lets you nest down to three layers; you want the function to be recursive, like in [my answer](http://stackoverflow.com/a/10218517/344821), to be able to keep going. – Danica Apr 18 '12 at 21:49
  • @Dougal: Your answer didn't say that when I wrote this. – MattH Apr 18 '12 at 21:52
  • @Dougal: If three levels deep is all the OP needs, it's sufficient. – Steven Rumbalski Apr 18 '12 at 21:54
  • @StevenRumbalski ...obviously, yes, but that's kind of a weird requirement considering that the OP only asked about "nested dicts". And yes, MattH, I edited my answer around the same time as you answered; my post was meant to be pointing out a drawback of your answer rather than saying "I already posted a better one you dummy." :p – Danica Apr 18 '12 at 21:58
  • This solution isn't wrong but I might need to go deeper than 3 levels, so Dougal's answer is better. Thanks! – skandocious Apr 18 '12 at 22:15
  • @skandocious: I'd advise caution in this approach. When I first stopped coding in perl (11+ years ago), I really missed dynamically defined arbitrarily deep datastructures for a couple weeks but I've never needed or used them since. – MattH Apr 18 '12 at 22:20
  • @MattH It's really just a matter of convenience in this case. As I jump around and fill my nested structure (in a couple of loops) it's nice to not have to continually check if a nested dict exists in every loop iteration before dropping a value into it. – skandocious Apr 18 '12 at 22:23