Understanding python's super method, Why D().test() will return 'B->C' and not 'B-->A'

Question

I have looked at other question here regarding python's super() method but I am still finding it difficult to understand the whole concept.

I am also looking at the example in the book pro python

The example referenced there is

class A(object):
     def test(self):
         return 'A'

class B(A):
     def test(self):
         return 'B-->' + super(B, self).test()

class C(A):
     def test(self):
         return 'C'

class D(B, C):
      pass

>>> A().test()
'A'
>>> B().test()
'B-->A'
>>> C().test()
'C'
>>> D().test()
'B-->C'

>>> A.__mro__
(__main__.A, object)

>>> B.__mro__
(__main__.B, __main__.A, object)

>>> C.__mro__
(__main__.C, __main__.A, object)

>>> D.__mro__
(__main__.D, __main__.B, __main__.C, __main__.A, object)

Why doing D().test() we get the output as 'B-->C' instead of 'B-->A'

The explanation in the book is

In the most common case, which includes the usage shown here, super() takes two arguments: a class and an instance of that class. As our example here has shown, the instance object determines which MRO will be used to resolve any attributes on the resulting object. The provided class determines a subset of that MRO, because super() only uses those entries in the MRO that occur after the class provided.

I still find the explanation a bit difficult to understand. This might be a possible duplicate and questions similar to this has been asked many times, but if I get an understanding of this I might be able to understand the rest of other questions better.

Understanding Python super() with __init__() methods

What does 'super' do in Python?

python, inheritance, super() method

[python]: confused by super()

Answer 1

If you want to know why Python chose this specific MRO algorithm, the discussion is in the mailing list archives, and briefly summarized in The Python 2.3 Method Resolution Order.

But really, it comes down to this: Python 2.2's method resolution was broken when dealing with multiple inheritance, and the first thing anyone suggested to fix it was to borrow the C3 algorithm from Dylan, and nobody had any problem with it or suggested anything better, and therefore Python uses C3.

---

If you're more interested in the general advantages (and disadvantages) of C3 against other algorithms…

BrenBarn's and florquake's answers give the basics to this question. Python's super() considered super! from Raymond Hettinger's blog is a much longer and more detailed discussion in the same vein, and definitely worth reading.

A Monotonic Superclass Linearlization for Dylan is the original paper describing the design. Of course Dylan is a very different language from Python, and this is an academic paper, but the rationale is still pretty nice.

Finally, The Python 2.3 Method Resolution Order (the same docs linked above) has some discussion on the benefits.

And you'd need to learn a lot about the alternatives, and about how they are and aren't appropriate to Python, to go any farther. Or, if you want deeper information on SO, you'll need to ask more specific questions.

---

Finally, if you're asking the "how" question:

When you call D().test(), it's obviously calling the code you defined in B's test method. And B.__mro__ is (__main__.B, __main__.A, object). So, how can that super(B, self).test() possibly call C's test method instead of A's?

The key here is that the MRO is based on the type of self, not based on the type B where the test method was defined. If you were to print(type(self)) inside the test functions, you'd see that it's D, not B.

So, super(B, self) actually gets self.__class__.__mro__ (in this case, D.__mro__), finds B in the list, and returns the next thing after it. Pretty simpler.

But that doesn't explain how the MRO works, just what it does. How does D().test() call the method from B, but with a self that's a D?

First, notice that D().test, D.test and B.test are not the same function, because they're not functions at all; they're methods. (I'm assuming Python 2.x here. Things are a little different—mainly simpler—in 3.x.)

A method is basically an object with im_func, im_class, and im_self members. When you call a method, all you're doing is calling its im_func, with its im_self (if not None) crammed in as an extra argument at the start.

So, our three examples all have the same im_func, which actually is the function you defined inside B. But the first two have D rather than B for im_class, and the first also has a D instance instead of None for im_self. So, that's how calling it ends up passing the D instance as self.

So, how does D().test end up with that im_self and im_class? Where does that get created? That's the fun part. For a full description, read the Descriptor HowTo Guide, but briefly:

Whenever you write foo.bar, what actually happens is equivalent to a call to getattr(foo, 'bar'), which does something like this (ignoring instance attributes, __getattr__, __getattribute__, slots, builtins, etc.):

def getattr(obj, name):
    for cls in obj.__class__.__mro__:
        try:
            desc = cls.__dict__[name]
        except KeyError:
            pass
        else:
            return desc.get(obj.__class__, obj)

That .get() at the end is the magic bit. If you look at a function—say, B.test.im_func, you'll see that it actually has a get method. And what it does is to create a bound method, with im_func as itself, im_class as the class obj.__class__, and im_self as the object obj.

Answer 2

The short answer is that the method resolution order is roughly "breadth first". That is, it goes through all the base classes at a given level of ancestry before going to any of their superclasses. So if D inherits from B and C, which both inherit from A, the MRO always has B and C before A.

Another way to think about it is that if the order went B->A, then A.test would be called before C.test, even though C is a subclass of A. You generally want a subclass implementation to be invoked before the superclass one (because the subclass one might want to totally override the superclass and not invoke it at all).

A longer explanation can be found here. You can also find useful information by googling or searching Stackoverflow for question about "Python method resolution order" or "Python MRO".

Answer 3

super() is basically how you tell Python "Do what this object's other classes say."

When each of your classes has only one parent (single inheritance), super() will simply refer you to the parent class. (I guess you've already understood this part.)

But when you use multiple base classes, as you did in your example, things start to get a little more complicated. In this case, Python ensures that if you call super() everywhere, every class's method gets called.

A (somewhat nonsensical) example:

class Animal(object):
    def make_sounds(self):
        pass

class Duck(Animal):
    def make_sounds(self):
        print 'quack!'
        super(Duck, self).make_sounds()

class Pig(Animal):
    def make_sounds(self):
        print 'oink!'
        super(Pig, self).make_sounds()

# Let's try crossing some animals
class DuckPig(Duck, Pig):
    pass

my_duck_pig = DuckPig()
my_duck_pig.make_sounds()
# quack!
# oink!

You would want your DuckPig to say quack! and oink!, after all, it's a pig and a duck, right? Well, that's what super() is for.