7

I want to define light-weight classes that are supposed to represent data structures. As is the case with many data structures, the order of the data is important. So if I go ahead and define this:

class User(DataStructure):
    username = StringValue()
    password = StringValue()
    age = IntegerValue()

I am implying that this is a data structure where a string with the username comes first, followed by a string with the password, and finally the age of the user as an integer.

If you're familiar with Python, you'll know that the above class, User, is an object inheriting from type. It will, just like most other objects in Python, have a __dict__. And here-in lies my problem. This __dict__ is a hash map, so the order of the class attributes in the __dict__ is in no way related to their order of definition.

Is there any way I can figure out the actual definition order? I'm asking here before I go with one of the less sane methods I can think of...

Oh and just to be clear, what I want is a way to get this out of the above definition: ['username', 'password', 'age']

Blixt
  • 49,547
  • 13
  • 120
  • 153
  • 1
    For reference, my "less sane" solution: Have a counter that goes up every time a `*Value` class is instantiated and set the new value to that instance, then order the list of attributes by this value. This would be done using a meta-class. – Blixt Nov 18 '10 at 23:34
  • Using a metaclass along the lines of what you've described here is the only thing that leapt to my mind. – Will McCutchen Nov 18 '10 at 23:40
  • well, the counter is the way that the Django team did it, so I'd be fairly sure that that's the best way to do it. – Chris Morgan Nov 18 '10 at 23:51

5 Answers5

5

Python 2.7 and 3.x define an OrderedDict in the collections module. I believe it uses a linked list to maintain insertion order of its items. It adds iterable methods to the standard mutable mapping methods.

You could define a metaclass which uses an OrderedDict rather than a standard unordered dict as the namespace __dict__ for your data structures classes. If you give your metaclass a special __prepare__() method you can do this. I haven't tried this but according to the docs it's possible:

From Python 3.1 Language Ref Section 3.3.3 Data Model - Customizing class creation:

If the metaclass has a __prepare__() attribute (usually implemented as a class 
or static method), it is called before the class body is evaluated with the 
name of the class and a tuple of its bases for arguments. It should return an 
object that supports the mapping interface that will be used to store the 
namespace of the class. The default is a plain dictionary. This could be used, 
for example, to keep track of the order that class attributes are declared in 
by returning an ordered dictionary.

Unfortunately the equivalent section 3.4.3 in the Python 2.7 Language Ref makes no mention of being able to substitute a class namespace dict and no mention of the __prepare__() method. So this may only be possible in Python version 3.

Don O'Donnell
  • 4,538
  • 3
  • 26
  • 27
4

This is something that is not well supported at all in Python. Django has employed metaclasses to deal with it. See this question: How does Django Know the Order to Render Form Fields?

(Summary: look at django.forms.forms.DeclarativeFieldsMetaclass, django.forms.forms.get_declared_fields and how creation_counter is used in django.forms.fields.Field.)

Community
  • 1
  • 1
Chris Morgan
  • 86,207
  • 24
  • 208
  • 215
  • Sounds like Django did what I was planning to do. I'll look at their code and also investigate Tobu's answer, at first glance it appears to be more correct. – Blixt Nov 18 '10 at 23:55
  • Tobu's and Don's answers would be great to implement, but are not supported in any of the Python 2.x versions. So for now, the Django solution is the best choice for me. Thanks! – Blixt Nov 19 '10 at 09:58
2

One way that would be more general than Django's approach, __slots__ and available in python 2 would be this metaclass:

class OrderedTypeMeta(type):
    def __new__(mcls, clsname, bases, clsdict):
        attrs = clsdict.get('_attrs_', [])
        attrnames = []
        for name, value in attrs:
            clsdict[name] = value
            attrnames.append(name) 
        clsdict['_attrs_'] = attrnames
        return super(OrderedTypeMeta, mcls).__new__(mcls, clsname, bases, clsdict)


class User(DataStructure):
    __metaclass__ = OrderedTypeMeta
    _attrs_ = (('name', StringValue()),
               ('password', StringValue()),
               ('age', IntegerValue()))

I say it's more general than django's way because you don't need the attributes to be instances of a particular class, any value will do. It's also more general than __slots__ because you will still be able to assign attributes to instances of the classes (though this may not be needed: in that case, I would prefer __slots__). In python3, I would prefer __prepare__.

The primary disadvantage of this, aside from the fact that it's sort of ugly, is that it will not work with inheritance. It would not be too difficult to get the __attrs__ out of the base classes and extend that instead of setting it to an empty list.

aaronasterling
  • 68,820
  • 20
  • 127
  • 125
  • 1
    It's a logically elegant solution, but the only reason I'm looking for a way to get the definition order is so that I wouldn't have make the syntax of the struct defs any more complex. So I'll probably go with the Django solution, since I actually have a particular base class for my structures. Thanks though! :) – Blixt Nov 19 '10 at 00:26
1

You could use __slots__ to explicitly order the attributes. If every class you inherit from defines it, there's a bonus performance gain on memory and attribute access.

Edit: The most reliable way would be to define __prepare__ in a metaclass. The reference docs have it, complete with an example of storing attributes in an OrderedDictionary.

You can't have the order implicitly in Python 2 without Django's field classes technique, because Python just doesn't track that information; it is lost by the time a metaclass is called.

Tobu
  • 24,771
  • 4
  • 91
  • 98
  • 2
    It should be noted that `__prepare__` is only available in python 3. – aaronasterling Nov 19 '10 at 00:07
  • Sorry, I forgot to say that I want this to work with Python 2.7. __prepare__ appears to be very new. Also, do you know of an implementation that can assign __slots__ dynamically? I tried a little and couldn't get it to work in 2.7. – Blixt Nov 19 '10 at 00:11
  • No, I was thinking "explicit is better than implicit". Put `__slots__` in the source, and either enforce that `__dict__` isn't defined or ignore non-slots attributes. – Tobu Nov 19 '10 at 00:25
  • Oh and thanks for the heads-up on `__prepare__`, I'll keep it in the back of my head until it's matured :) – Blixt Nov 19 '10 at 00:28
  • Yeah I understand that's what you meant. I'm actually looking for implicit over explicit in this particular case since the classes are configuration classes and should be a "define everything once" kind of deal, so they can be updated often without error. Adding `__slots__` in the mix would work against that idea. – Blixt Nov 19 '10 at 00:29
  • 1
    My approach to having things updated without error is to bomb out the first time the erroneous code is introduced. Forgot to order the new attr? No class for you. – Tobu Nov 19 '10 at 00:54
  • Code should fail early. I totally agree. But, adding `__slots__` to the definition *still* adds a point of failure, and extra manual work to keep two lists in sync. I'm willing to sacrifice the performance loss to make code simpler for developers to maintain. Coder performance > code performance. It's the reason I'm using Python in the first place. :) – Blixt Nov 19 '10 at 09:43
0

The following let's me capture the order of attributes that are methods or of type struct. I have not tried it with builtin types. You can then build a walker around children

class ordered_type(type):
    __ordernate__ = 0
    def __new__(meta, name, baseclasses, dct):
        new = super(Type, meta).__new__(meta, name, baseclasses, dct)
        new.__ordernate__ = Type.__ordernate__
        Type.__ordernate__ += 1        
        def key(x):
            from inspect import ismethod
            if ismethod(x):
                return x.im_func.__code__.co_firstlineno
            elif isinstance(x, type):
                return x.__ordernate__ + 1000000
        new.__children__ = sorted(
            [getattr(new, x) for x in dir(new) if not x.startswith('__')],
            key=key)

class struct(object):
    __metaclass__ = ordered_type

#use case
class A(struct):
    '''Description of class A
    '''
    def name(self):
        '''The name of instance of A
        '''
    class B(struct):
        '''Description of class B
        '''
        def name(self):
            '''The name of instance of B
            '''