4

I have two classes in my code. first is the parent, which second inherits.

class first(object):
    def __init(self,**kwargs):  
        pass

    def __setattr__(self,name,value):
        self.__dict__[name] = value

class second(first):
    def do_something(self):
        self.a = 1
        self.b = 2
        self.c = 3

when I am printing the class second (by e.g. second.__dict__) I get the unordered dictionary. This is obvious. I want to change this behavior to get an ordered dictionary using the OrderedDict class, but it does not work. I am changing implementation of first in the following way:

class first(OrderedDict):   
    def __init__(self,**kwargs):  
        super(first,self).__init__(**kwargs)  
    def __setattr__(self,name_value):  
        super(first,self).__setattr__(name_value)  

I would like to print second using __dict__ or __repr__, but I got the unordered dictionary. What should I change?

jonrsharpe
  • 115,751
  • 26
  • 228
  • 437
Przemo
  • 193
  • 2
  • 16
  • 8
    The `__dict__` of `OrderDict` instances is not itself an `OrderedDict` instance... – Martijn Pieters Jan 14 '15 at 11:15
  • 1
    Could you explain *why* you want this? There might be other approaches you could explore. – jonrsharpe Jan 14 '15 at 11:28
  • Why do you _need_ the `__dict__` to be ordered? Plain `dict`s are more efficient, and for printing you can simply sort the dict in your `__repr__` or `__str__` methods. – PM 2Ring Jan 14 '15 at 11:28
  • I would like to have an order of elements the same as in moment of creating them. I print such info on the screen and order matters. Sorting in this case does not help. – Przemo Jan 14 '15 at 12:32
  • @PM 2Ring: The `OrderDict` items might need to be kept is some order that can't be simulated by simply sorting them. One real-world example might be to simply maintain the order in which they were added/created. I suppose that could simulated, too, but the point is that it shouldn't take that much effort to do so when an `OrderedDict` already can (and is likely much faster). – martineau Jul 11 '18 at 21:13
  • 1
    @martineau Fair point. But of course this is a non-issue for Python 3.6+, since plain `dict` now maintains insertion order. – PM 2Ring Jul 12 '18 at 14:31
  • Yes this question is obsolete in 3.7 [Are dictionaries ordered in Python 3.6+?](https://stackoverflow.com/questions/39980323/are-dictionaries-ordered-in-python-3-6) – smci Jan 06 '19 at 11:27

4 Answers4

11

You can simply redirect all attribute access to an OrderedDict:

class first(object):
    def __init__(self, *args, **kwargs):  
        self._attrs = OrderedDict(*args, **kwargs)

    def __getattr__(self, name):
        try:
            return self._attrs[name]
        except KeyError:
            raise AttributeError(name)

    def __setattr__(self, name, value):
        if name == '_attrs':
            return super(first, self).__setattr__(name, value)
        self._attrs[name] = value

Demo:

>>> from collections import OrderedDict
>>> class first(object):
...     def __init__(self, *args, **kwargs):  
...         self._attrs = OrderedDict(*args, **kwargs)
...     def __getattr__(self, name):
...         try:
...             return self._attrs[name]
...         except KeyError:
...             raise AttributeError(name)
...     def __setattr__(self, name, value):
...         if name == '_attrs':
...             return super(first, self).__setattr__(name, value)
...         self._attrs[name] = value
... 
>>> class second(first):
...     def do_something(self):
...         self.a = 1
...         self.b = 2
...         self.c = 3
... 
>>> s = second()
>>> s.do_something()
>>> s._attrs
OrderedDict([('a', 1), ('b', 2), ('c', 3)])

You can't otherwise replace the __dict__ attribute with an OrderedDict instance, because Python optimises instance attribute access by using the concrete class API to access the dictionary internals in C, bypassing the OrderedDict.__setitem__ hook altogether (see issue #1475692).

Martijn Pieters
  • 1,048,767
  • 296
  • 4,058
  • 3,343
  • Thank you very much for this demo. In fact it works in this simple example, but after implementing this in my freaky code I got an error: RuntimeError: maximum recursion depth exceeded. i am looking for the bug in code, but in the meantime maybe you could find the reason of this issue? Or propose alternative solution to avoid such an issue. I would like to not increment recursion depth. – Przemo Jan 14 '15 at 12:40
  • @Przemo: your `__setattr__` is calling itself perhaps? How are you using `super()` here? Make sure you guard against setting `_attrs` in the wrong object as well. – Martijn Pieters Jan 14 '15 at 12:42
  • @Przemo: increasing the recursion depth is not going to solve this issue, you have an *infinite* loop there. – Martijn Pieters Jan 14 '15 at 12:42
  • I got one calling of __setattr__ method and then many of callings of __getattr__. It will try with your suggestion. Thanks. – Przemo Jan 14 '15 at 12:50
  • @Przemo: right, that means you do not yet have a specific attribute that `__setattr__` tried to access. Your `_attrs` attribute is probably missing, so `self._attrs[name]` calls `__getattr__('_attrs)`, which tries to use `self._attrs[name]`, which doesn't exist, so it calls `__getattr__`, etc. – Martijn Pieters Jan 14 '15 at 12:53
  • @Przemo: this is why my `__setattr__` method explicitly tests for `_attrs` *first* and makes sure it is set by calling the original version via `super()`. – Martijn Pieters Jan 14 '15 at 12:54
  • I'd drop the `**kwargs` as it packs things into a regular dict in between. `first(**OrderedDict([...]))` looses the order. – mdaoust Nov 17 '15 at 17:37
  • 1
    @Suki: how about using `*args`, in addition? – Martijn Pieters Nov 17 '15 at 17:45
  • @Martijn great, that way the call signature matches an `OrderedDict`. – mdaoust Nov 17 '15 at 19:37
  • Seems like this approach would generally slow down _all_ attribute access because now they require either a `__getattr__()` or `__setattr__()` call. – martineau Jul 10 '18 at 23:36
  • @martineau: which is why you can't just replace the `__dict__` attribute either (which would require Python to call `__setitem__` on whatever replaced the `dict` instance currently used). – Martijn Pieters Jul 11 '18 at 09:24
  • Martijn: Hmm...I understand the point you're making (and have read the linked bug-report which indicates the issue is still open), However, if you actually try it out, it seems that one **_can_** just replace `__dict__` with an `OrderdDict` in the class' `__init__` method as shown in @Kresimir's [answer](https://stackoverflow.com/a/32699282/355230)—at least with Python 2.7.14 and 3.6.5. What am I missing? – martineau Jul 11 '18 at 18:03
  • @martineau: sure you can, but you can't actually set attributes *and store them in the `OrderedDict` in a way that keeps them ordered*. – Martijn Pieters Jul 11 '18 at 20:09
  • 1
    @martineau: what happens is that `OrderedDict` is a subclass of `dict`, and the normal `__getattribute__` and `__setattr__` implementation for instances (in C) just reaches into the C-level data structures. The custom `__setitem__` method of the `OrderedDict` class is not consulted. When you actually print the `__dict__` reference you'll be shown an empty `OrderedDict` instance, because all the internal order-recording structure is not being maintained. `list(d.__dict__)` is empty, so is `list(d.__dict__.items())`, and only `len(d.__dict__)` is correct. The keys are stored though. – Martijn Pieters Jul 11 '18 at 20:20
  • Martijn: Ah, OK, now I get it. Note however if I `print(test.__dict__)` the result shown is not empty—it's `OrderedDict([('a', 0), ('b', 1), ('c', 2)])`—and the attribute order is maintained after a `test.a, test.b, test.c = 'a', 'b', 'c'` assignment. However _adding_ a new item with `test.foo = 42` doesn't change it contents. Thanks for bearing with me in making sense of the subject. I would think a metaclass could probably get around the problem caused by the "issue" (which the Python devs seem in no hurry to fix...) – martineau Jul 11 '18 at 20:54
  • @MartijnPieters it is 2022 now and since python 3.6 the __dict__ is a OrderedDict by default. In addition someone could use `__prepare__` nowadays. Would you mind updating your answer? – Thingamabobs Jun 19 '22 at 07:34
  • @Thingamabobs: no, `__dict__` is a `dict` object, not an `OrderedDict`. `dict` preserves insertion order, but **is not reorderable**, where `OrderedDict` *can* be reordered. There are still cases where you want an `OrderedDict` instead of the base `dict`. – Martijn Pieters Jul 07 '22 at 13:07
5

I think the solutions in this thread focus too much on using OrderedDict as if it is a necessity. The class already has a builtin __dict__ method, the only problem is ordering the keys. Here is how I am retrieving (key, value) pairs from my class in the order they are entered:

class MyClass:

    def __init__(self, arg1, arg2, arg3):
        self._keys = []
        self.arg1 = arg1
        self.arg2 = arg2
        self.arg3 = arg3

    def __setattr__(self, key, value):
        # store new attribute (key, value) pairs in builtin __dict__
        self.__dict__[key] = value
        # store the keys in self._keys in the order that they are initialized
        # do not store '_keys' itelf and don't enter any key more than once 
        if key not in ['_keys'] + self._keys:
            self._keys.append(key)

    def items(self):
        # retrieve (key, value) pairs in the order they were initialized using _keys
        return [(k, self.__dict__[k]) for k in self._keys]

>>> x = MyClass('apple', 'orange', 'banana')
>>> print x.items()
[('arg1', 'apple'), ('arg2', 'orange'), ('arg3', 'banana')]
>>> x.arg1 = 'pear'
>>> print x.items()
[('arg1', 'pear'), ('arg2', 'orange'), ('arg3', 'banana')]

I'm using a class to store about 70 variables used to configure and run a much bigger program. I save a text copy of the initial (key, value) pairs that can be used to initialize new instances of the class. I also save a text copy of the (key, value) pairs after running the program because several of them are set or altered during the program run. Having the (key, value) pairs in order simply improves the readability of the text file when I want to scan through the results.

isosceleswheel
  • 1,516
  • 12
  • 20
  • 1
    This approach would also work well for pre-2.7 versions of Python that don't have a built-in `collections.OrderedDict`. Store new values wouldn't be quite a fast, since doing so now requires a linear search...on the other hand, reading values of existing attribute would be just a fast as it normally would be (i.e. doesn't force all accesses to go through `__getattr__()` or `__setattr__()` calls. – martineau Jul 10 '18 at 23:49
1

You can try by actually replacing __dict__ with OrderedDict:

from collections import OrderedDict

class Test(object):
    def __init__(self):
        self.__dict__ = OrderedDict()
        self.__dict__['a'] = 0
        self.__dict__['b'] = 1
        self.__dict__['c'] = 2

test = Test()
print test.__dict__
test.a, test.b, test.c = 'a', 'b', 'c'
print test.__dict__

This should printout:

OrderedDict([('a', 0), ('b', 1), ('c', 2)])
OrderedDict([('a', 'a'), ('b', 'b'), ('c', 'c')])
Kresimir
  • 2,930
  • 1
  • 19
  • 13
  • This looks like a bad idea. I don't know exactly what's happening, but i think the `OrderedDict` is ending up in the `'__dict__'` slot of the real `__dict__`, and all kinds of unpredictable nonsense is happening. Try it again without the `self.__dict__['a'] = 0` lines and see. – mdaoust Nov 17 '15 at 17:40
  • 1
    @mdaoust: Replacing the instance's `__dict__` with another mapping object is fine. See the [documentation](https://docs.python.org/2/library/stdtypes.html#object.__dict__). – martineau Jun 16 '17 at 15:37
  • 1
    @mdaoust: Sorry, my earlier comment that doing something like was fine was **incorrect**. See the end of Martijn Pieters' [answer](https://stackoverflow.com/a/27941731/355230) (and the comments under it) for more details. – martineau Jul 11 '18 at 21:02
  • @mdaoust Actually I think your earlier comment is right. The [issue](https://github.com/python/cpython/issues/43272) is unrelated, it just says the __getitem__ is not correctly recognized in attribute lookup. [My code](https://github.com/ZhiyuanChen/DanLing/blob/develop/danling/runner/runner.py) works perfect – Zhiyuan Chen Dec 12 '22 at 14:55
-1

Another option; you can also manipulate new if you wish.

from collections import OrderedDict
class OrderedClassMeta(type):
    @classmethod
        def __prepare__(cls, name, bases, **kwds):
        return OrderedDict()
class OrderedClass(metaclass=OrderedClassMeta):
    pass

class A(OrderedClass):
    def __init__(self):
        self.b=1
        self.a=2
    def do(self):
        print('do')
class B(OrderedClass):
    def __init__(self):
        self.a=1
        self.b=2
    def do(self):
        print('do')

 a=A()
 print(a.__dict__)
 b=B()
 print(b.__dict__)
Arnon Sela
  • 31
  • 2
  • Please try your answer before posting, it doesn't work. The return value of `__prepare__` gets copied to a standard dict in C. See https://mail.python.org/pipermail/python-list/2012-January/618121.html. – Henry Schreiner Jul 28 '15 at 21:42
  • There was indentation issue the original post (for "def __prepare__"). If you clean the indentation, and using Python 3.6, the result look as follows: >>> a=A() >>> print(a.__dict__) {'b': 1, 'a': 2} >>> b=B() >>> print(b.__dict__) {'a': 1, 'b': 2} >>> – Arnon Sela Aug 11 '17 at 03:17