Why can't I create a default, ordered dict by inheriting OrderedDict and defaultdict?

Question

My first attempt to combine the features of two dictionaries in the collections module was to create a class that inherits them:

from collections import OrderedDict, defaultdict

class DefaultOrderedDict(defaultdict, OrderedDict):
    def __init__(self, default_factory=None, *a, **kw):
        super().__init__(default_factory, *a, **kw)

However, I cannot assign an item to this dictionary:

d = DefaultOrderedDict(lambda: 0)
d['a'] = 1
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib64/python3.3/collections/__init__.py", line 64, in __setitem__
    self.__map[key] = link = Link()
AttributeError: 'DefaultOrderedDict' object has no attribute '_OrderedDict__map'

Indeed, this question about how to create a similar object has answers that achieve it by extending the OrderedDict class and manually re-implementing the additional methods provided defaultdict. Using multiple inheritance would be cleaner. Why doesn't it work?

As an aside, you can use `defaultdict(int)` instead of `defaultdict(lambda: 0)`. — 1'', Apr 08 '15 at 16:18

Ashwini Chaudhary · Answer 1 · 2014-12-30T21:41:56.887

The reason is that the init method of defaultdict instead of calling __init__ of the next class in MRO calls init of PyDict_Type hence some of the attributes like __map that are set in OrderedDict's __init__ are never initialized, hence the error.

>>> DefaultOrderedDict.mro()
[<class '__main__.DefaultOrderedDict'>,
 <class 'collections.defaultdict'>,
 <class 'collections.OrderedDict'>,
 <class 'dict'>, <class 'object'>]

And defaultdict don't have their own __setitem__ method:

>>> defaultdict.__setitem__
<slot wrapper '__setitem__' of 'dict' objects>
>>> dict.__setitem__
<slot wrapper '__setitem__' of 'dict' objects>
>>> OrderedDict.__setitem__
<unbound method OrderedDict.__setitem__>

So, when you called d['a'] = 1, in search of __setitem__ Python reached OrdereredDict's __setitem__ and their the access of uninitialized __map attribute raised the error:

A fix will be to call __init__ on both defaultdict and OrderedDict explicitly:

class DefaultOrderedDict(defaultdict, OrderedDict):
    def __init__(self, default_factory=None, *a, **kw):
        for cls in DefaultOrderedDict.mro()[1:-2]:
            cls.__init__(self, *a, **kw)

Why can't you just swap the order of the base classes: `class OrderedDefaultDict(OrderedDict, defaultdict):`, and then have your `__init__(self, default_factory=None, *args, **kwargs)` contain two lines: `super(OrderedDefaultDict, self).__init__(*args, **kwargs)` and `self.default_dict = default_dict`? — Sam, Jul 03 '15 at 18:38
I just tried Sam's suggestion and it worked. (Well, after I changed default_dict to default_factory!) — samwyse, Dec 18 '15 at 12:31

grasshopper · Answer 2 · 2014-12-30T21:12:02.137

Perhaps you are coming from a Java background, but multiple inheritance doesn't do what you'd expect it does in Python. Calling super from the init of the defaultOrderedDict calls the super() as the init of defaultdict and never the init of OrderedDict. The map attribute is first defined in the __init function of OrderedDict. The implementation is the following (from source):

def __init__(self, *args, **kwds):
    '''Initialize an ordered dictionary.  The signature is the same as
    regular dictionaries, but keyword arguments are not recommended because
    their insertion order is arbitrary.

    '''
    if len(args) > 1:
        raise TypeError('expected at most 1 arguments, got %d' % len(args))
    try:
        self.__root
    except AttributeError:
        self.__root = root = []                     # sentinel node
        root[:] = [root, root, None]
        self.__map = {}
    self.__update(*args, **kwds)

Note that this doesn't have to do with the attribute being private. A minimal example with multiple inheritance can illustrate this:

class Foo:
    def __init__(self):
        self.foo=2

class Bar:
    def __init__(self):
        self.bar=1

class FooBar(Foo,Bar):
     def __init__(self):
        super().__init__()

fb = FooBar()

fb.foo
>>2 
fb.bar
>>AttributeError: 'FooBar' object has no attribute 'bar'

So, the constructor of Bar was never called. Pythons method resolution order goes from left to right until it finds a class with the function name it seeks (in this case init) and then ignores all other classes on the right (in this case Bar)

Why can't I create a default, ordered dict by inheriting OrderedDict and defaultdict?

2 Answers2

Linked