How does multiple inheritance work in descriptors?

Question

I watched a great video on YouTube about Python metaprogramming. I tried to write the following code (which is almost the same from the video):

class Descriptor:
    def __init__(self, name):
        self.name = name

    def __get__(self, instance, cls):
        return instance.__dict__[self.name]

    def __set__(self, instance, val):
        instance.__dict__[self.name] = val

    def __delete__(self, instance):
        del instance.__dict__[self.name]

class Type(Descriptor):
    ty = object
    def __set__(self, instance, val):
        if not isinstance(val, self.ty):
            raise TypeError("%s should be of type %s" % (self.name, self.ty))
        super().__set__(instance, val)

class String(Type):
    ty = str

class Integer(Type):
    ty = int

class Positive(Descriptor):
    def __set__(self, instance, val):
        if val <= 0:
            raise ValueError("Must be > 0")
        super().__set__(instance, val)

class PositiveInteger(Integer, Positive):
    pass

class Person(metaclass=StructMeta):
    _fields = ['name', 'gender', 'age']
    name = String('name')
    gender = String('gender')
    age = PositiveInteger('age')

So PositiveInteger is inherited from Integer and Positive, and both classes have __get__ method defined to do some validation. I wrote some test code to convince myself that both methods will run:

class A:
    def test(self):
        self.a = 'OK'

class B:
    def test(self):
        self.b = 'OK'

class C(A, B):
    pass

c = C()
c.test()
print(self.a)
print(self.b)

Only to find that only the first print statement works. The second will raise an AttributeError, which indicates that when there's name conflict, the first base class wins.

So I wonder why both validations work? It's even more weird that when only the Integer check passes (e.g. person.age = -3), it's super().__set__(instance, val) has no effect, leaving person.age untouched.

score 2 · Accepted Answer · edited May 23 '17 at 10:26

The validation logic of both Positive and Integer runs because both Type and Positive have this line in __set__:

super().__set__(instance, val)

This doesn't skip to Descriptor.__set__. Instead, it calls the next method in method resolution order. Type.__set__ gets called, and its super().__set__(instance, val) calls Positive.__set__. Positive.__set__ runs its validation and calls Descriptor.__set__, which does the setting. This behavior is one of the reasons we have super.

If you wanted your test methods to behave like that, you would need to do two things. First, you would need to make A and B inherit from a common base class with a test method that doesn't do anything, so the super chains end at a place with a test method instead of going to object:

class Base:
    def test():
        pass

Then, you would need to add super().test() to both A.test and B.test:

class A(Base):
    def test(self):
        self.a = 'OK'
        super().test()

class B(Base):
    def test(self):
        self.b = 'OK'
        super().test()

For more reading, see Python's super() considered super.

Thanks for your excellent explanation! I was posting my own answer and didn't know someone has already answered this question until I refreshed the page. :) — Yukio Usuzumi, May 25 '14 at 12:29

Yukio Usuzumi · Answer 2 · 2014-05-25T12:34:21.413

Sorry, my bad.

The video gave perfect explanation just minute after where I paused and asked this question.

So when multiple inheritance happends, there's MRO thing (Method Resolution Order) defined in each class that determines the resolution order of methods in the super() chain. The order is determined by depth-first search, e.g.

class A:
    pass
class B(A):
    pass
class C(B):
    pass
class D(A):
    pass
class E(C, D):
    pass

E.__mro__ will be:

(<class '__main__.E'>, <class '__main__.C'>, <class '__main__.B'>, <class '__main__.D'>, <class '__main__.A'>, <class 'object'>)

One thing to notice is that A will appear in the inheritance tree multiple times, and in the MRO list it will only be in the last place where all A's appear.

Here's the trick: the call to super() won't necessarily go to its base. Instead, it'll find in the MRO list what comes next.

So to explain what happens in the code: The super() call in Integer.__get__ (which is inherited from Type.__get__) won't go to Descriptor.__get__, because Descriptor appears last in the MRO list. It will fall into Positive.__set__, and then its super() will fall into Descriptor, which will eventually set the value of the attribute.

How does multiple inheritance work in descriptors?

2 Answers2