Python dict using dot notation and chaining

Question

Ideally what I am aiming to accomplish is a class that extends (or is very similar to) a dict in Python with the additional capabilities:

Dot-Notation capable for setting and getting values
Key-Value capabilities like dict (i.e. setitem,getitem)
Can chain dot-notated operations

The goal is if I have something like example = DotDict() I could do the following against it example.configuration.first= 'first' and it would instantiate the appropriate DotDict instances under example with the really painful caveat being that if the operation is not assignment it should simply raise a KeyError like a dict would do

Here is what I have naively assembled

class DotDict(dict):
    def __getattr__(self, key):
        """ Make attempts to lookup by nonexistent attributes also attempt key lookups. """
        import traceback
        import re
        s= ''.join(traceback.format_stack(sys._getframe(1),1))
        if re.match(r'  File.*\n.*[a-zA-Z]+\w*\.[a-zA-Z]+[a-zA-Z0-9_. ]*\s*=\s*[a-zA-Z0-9_.\'"]+',s):
            self[key] = DotDict()
            return self[key]

        return self[key]

    def __setattr__(self, key, value):
        if isinstance(value,dict):
            self[key] = DotDict(value)
        self[key] = value

It works except for some common edge cases, I must say that I absolutely hate this method and there must be a better way. Looking at the stack and running a regular expression on the last line is not a good way to accomplish this.

The heart of the matter is that Python interprets lines of code left to right so when it arrives at a statement like a.b.c = 3 it's first operation is a getattr(a,b) and not a setattr so I can't determine easily if the last operation in the stack of operations is an assignment.

What I would like to know is if there is a good way to determine the last operation in the stack of operations or at least if it's a setattr.

Edit:

This is the solution that I came up with thanks to user1320237's recommendation.

class DotDict(dict):
    def __getattr__(self, key):
        """ Make attempts to lookup by nonexistent attributes also attempt key lookups. """
        if self.has_key(key):
            return self[key]
        import sys
        import dis
        frame = sys._getframe(1)
        if '\x00%c' % dis.opmap['STORE_ATTR'] in frame.f_code.co_code:
            self[key] = DotDict()
            return self[key]

        raise AttributeError('Problem here')

    def __setattr__(self, key, value):
        if isinstance(value,dict):
            self[key] = DotDict(value)
        self[key] = value

There's a little bit more in the actual implementation but it does an awesome job. The way it works is that it inspects the last frame in the stack and checks the byte code for a STORE_ATTR operation which means that the operation being performed is of the a.b.this.doesnt.exist.yet = 'something' persuasion. I would be curious if this could be done on other interpreters outside of CPython.

Do raise `AttributeError` instead of `KeyError`, otherwise your code wouldn't work with `hasattr` and such. — Antti Haapala -- Слава Україні, Aug 14 '14 at 07:04
Related: [Set Python dict items recursively, when given a compound key 'foo.bar.baz'](https://stackoverflow.com/questions/48729220/set-python-dict-items-recursively-when-given-a-compound-key-foo-bar-baz) — smci, Jan 06 '19 at 10:56

score 3 · Accepted Answer · edited Oct 19 '12 at 08:10

3

You may need to overwrite getattribute for those edge cases and then use the

object.__getattribute__

Have a look at the module dis. But what you wrote is nicer than disassembling.

>>> import dis
>>> def g():
    a.b.c = 4


>>> dis.dis(g)
  2           0 LOAD_CONST               1 (4)
              3 LOAD_GLOBAL              0 (a)
              6 LOAD_ATTR                1 (b)
              9 STORE_ATTR               2 (c)
             12 LOAD_CONST               0 (None)
             15 RETURN_VALUE

edited Oct 19 '12 at 08:10

Michael Allan Jackson

4,217
3
35
45

answered May 03 '12 at 20:49

User

14,131
2
40
59

Thanks, that looks great, I'm gonna play around with that a bit and in the mean time give you the check! – lukecampbell May 03 '12 at 20:52
I can't begin to tell you how much use I'm already getting out of this, thank you! Check out the basic implementation above. – lukecampbell May 04 '12 at 14:52

rv.kvetch · Answer 2 · 2022-06-23T20:52:48.937

In case anyone is re-visiting this question in 2022, here's the version I found that works in Python 3 -- at least, I was able to test this on my Mac machine which is running Python 3.10.

Note that the code from the original question requires a slight modification in 3.x apparently, since dict.has_key appears to be deprecated now, among a few other reasons.

import sys
import dis


_STORE_ATTR_OP = dis.opmap['STORE_ATTR']


class DotDict(dict):

    def __getattr__(self, key, __get=dict.__getitem__):
        """ Make attempts to lookup by nonexistent attributes also attempt key lookups. """
        if key in self:
            return __get(self, key)

        frame = sys._getframe(1)

        if _STORE_ATTR_OP in frame.f_code.co_code:
            self[key] = res = DotDict()
            return res

        raise AttributeError('Problem here')

    def __setattr__(self, key, value):
        self[key] = DotDict(value) if isinstance(value, dict) else value

NB: I've also published a library on PyPI in case anyone is curious, dotwiz, which provides it's own version of DotDict essentially.

The main impetus with this approach is performance, so __setattr__, and especially __getitem__ or access times should be really fast -- just as fast as a plain dict really.

I'm also planning to add support for a "chaining" approach with __setitem__ as is inspired by this question, so for example:

obj['a.b.c.d'] = 1

should then work, and essentially perform the equivalent of:

obj = ✫(a=✫(b=✫(c=✫(d=1))))

And maybe it makes sense to add a fallback method, set, so that can be used for cases where we don't actually desire that behavior. So in the example above,

obj.set('a.b.c', 2)

should result in the following value of obj:

✫(a.b.c=2)

score 1 · Answer 3 · answered May 03 '12 at 21:15

Based on user1320237's answer, here's what I got. It seems to do what you want it to.

class A(object):
    def __getattribute__(self, attr):
        try:
            return super(A, self).__getattribute__(attr)
        except AttributeError:
            self.__setattr__(attr, A())
            return super(A, self).__getattribute__(attr)

score 1 · Answer 4 · answered Aug 21 '13 at 07:22

Here is another solution: http://tech.zarmory.com/2013/08/python-putting-dot-in-dict.html

>>> d = DefaultDotDict({1: {2: 3}})
>>> d.a.b.c.d = "magic!"
>>> import json; print json.dumps(d, indent=2)
{
  "a": {
    "b": {
      "c": {
        "d": "magic!"
      }
    }
  }, 
  "1": {
    "2": 3
  }
}
>>>

FreeFox · Answer 5 · 2016-11-16T06:24:17.153

-2

class DotDict(dict):

"""Dot natation for dict"""

def __init__(self, theDict):
    super(MyDict, self).__init__()
    for key, item in theDict.items():
        if isinstance(item, dict):
            item = MyDict(item)
        self.__dict__[key] = item

def __getattr__(self, key):
    return self.__dict__[key]

def __setattr__(self, name, value):
    self.__dict__[name] = value

edited Nov 16 '16 at 06:24

answered Nov 16 '16 at 06:17

FreeFox

1
2

`self.__dict__[key] = item` **no** Don't do this. – juanpa.arrivillaga Jun 23 '22 at 21:08

Python dict using dot notation and chaining

5 Answers5

Linked