4

The document about super() found on the Python website says it returns a proxy object that delegates method calls to parent or sibling class. Information found at super considered super, how does super() work with multiple inheritance and super considered harmful explains that in fact the next method in the mro is used. My question is, what happens if super(object, self).some_method() is used? Since object typically appears at the end of a mro list, I guess the search will hit the end immediately with an exception. But in fact, it seems that the methods of the proxy itself were called, as shown by super(object, self).__repr__() showing the super object itself. I wonder whether the behavior of super() with object is not to delegate method at all. If this is the case, I wonder any reliable material ever mentions it and whether it applies to other Python implementations.

class X(object):
    def __init__(self):
        # This shows [X, object].
        print X.mro()

        # This shows a bunch of attributes that a super object can have.
        print dir(super(object, self))

        # This shows something similar to <super object at xxx>
        print(object, self)

        # This failed with `super() takes at least one argument`
        try:
            super(object, self).__init__()
        except:
            pass

        # This shows something like <super <class 'object'>, <'X' object>>.
        print(super(object, self).__repr__())

        # This shows the repr() of object, like <'X' object at xxx>
        print(super(X, self).__repr__())


if __name__ == '__main__':
    X()
cgsdfc
  • 538
  • 4
  • 19
  • Yes I know `super()` can call methods of sibling of `object`, but this cannot change the fact that `object` appears at the very end of mro list except when old-style class is present. – cgsdfc Jun 02 '18 at 12:18

2 Answers2

2

If super doesn't find something while looking through the method resolution order (MRO) to delegate to (or if you're looking for the attribute __class__) it will check its own attributes.

Because object is always the last type in the MRO (at least to my knowledge it's always the last one) you effectively disabled the delegation and it will only check the super instance.


I found the question really interesting so I went to the source code of super and in particular the delegation part (super.__getattribute__ (in CPython 3.6.5)) and I translated it (roughly) to pure Python accompanied by some additional comments of my own:

class MySuper(object):
    def __init__(self, klass, instance):
        self.__thisclass__ = klass
        self.__self__ = instance
        self.__self_class__ = type(instance)

    def __repr__(self):
        # That's not in the original implementation, it's here for fun
        return 'hoho'

    def __getattribute__(self, name):
        su_type = object.__getattribute__(self, '__thisclass__')
        su_obj = object.__getattribute__(self, '__self__')
        su_obj_type = object.__getattribute__(self, '__self_class__')

        starttype = su_obj_type

        # If asked for the __class__ don't go looking for it in the MRO!
        if name == '__class__':
            return object.__getattribute__(self, '__class__')
        mro = starttype.mro()
        n = len(mro)

        # Jump ahead in the MRO to the passed in class 
        # excluding the last one because that is skipped anyway.
        for i in range(0, n - 1):
            if mro[i] is su_type:
                break
        # The C code only increments by one here, but the C for loop
        # actually increases the i variable by one before the condition
        # is checked so to get the equivalent code one needs to increment
        # twice here.
        i += 2
        # We're at the end of the MRO. Check if super has this attribute.
        if i >= n:
            return object.__getattribute__(self, name)

        # Go up the MRO
        while True:
            tmp = mro[i]
            dict_ = tmp.__dict__
            try:
                res = dict_[name]
            except:
                pass
            else:
                # We found a match, now go through the descriptor protocol
                # so that we get a bound method (or whatever is applicable)
                # for this attribute.
                f = type(res).__get__
                f(res, None if su_obj is starttype else su_obj, starttype)
                res = tmp
                return res

            i += 1
            # Not really the nicest construct but it's a do-while loop
            # in the C code and I feel like this is the closest Python
            # representation of that.
            if i < n:
                continue
            else:
                break

        return object.__getattribute__(self, name)

As you can see there are some ways you could end up looking up the attribute on super:

  • If you're looking for the __class__ attribute
  • If you reached the end of the MRO immediately (by passing in object as first argument)!
  • If __getattribute__ couldn't find a match in the remaining MRO.

Actually because it works like super you can use that instead (at least as far as the attribute delegation is concerned):

class X(object):
    def __init__(self):
        print(MySuper(object, self).__repr__())

X()

That will print the hoho from the MySuper.__repr__. Feel free to experiment with that code by inserting some prints to follow the control flow.

I wonder any reliable material ever mentions it and whether it applies to other Python implementations.

What I said above was based on my observations of the CPython 3.6 source, but I think it shouldn't be too different for other Python versions given that the other Python implementations (often) follow CPython.

In fact I also checked:

And all of them return the __repr__ of super.

Note that Python follows the "We are all consenting adults" style, so I would be surprised if someone bothered to formalize such unusual usages. I mean who would try to delegate to a method of the sibling or parent class of object (the "ultimate" parent class).

MSeifert
  • 145,886
  • 38
  • 333
  • 352
  • Well, very in-depth research. I gain a better understanding about the attribute lookup mechanism. Thanks. – cgsdfc Jun 03 '18 at 04:31
0

super defines a few of its own attributes and needs a way to provide access to them. First is uses the __dunder__ style, which Python reserves for itself and says no library or application should define names that start and end with a double underscore. This means the super object can be confident that nothing will clash with its attributes of __self__, __self_class__ and __thisclass__. So if it searches the mro and doesn't find the requested attribute then it falls back on trying to find the attribute on the super object itself. For instance:

>>> class A:
    pass

>>> class B(A):
    pass

>>> s = super(A, B())
>>> s.__self__
<__main__.B object at 0x03BE4E70>
>>> s.__self_class__
<class '__main__.B'>
>>> s.__thisclass__
<class '__main__.A'>

Since you have specified object as the type to start looking beyond and because object is always the last type in the mro, then there is no possible candidate for which to fetch the method or attribute. In this situation, super behaves as if it had tried various types looking for the name, but didn't find one. So it tries to fetch the attribute from itself. However, since the super object is also an object it has access to __init__, __repr__ and everything else object defines. And so super returns its own __init__ and __repr__ methods for you.

This is kind of a situation of "ask a silly question (of super) and get a silly answer". That is, super should only ever be called with the first argument as class that the function was defined in. When you call it with object you are getting undefined behaviour.

Dunes
  • 37,291
  • 7
  • 81
  • 97
  • Are you sure `super` traverses the MRO first before returning one of its own attributes? That sounds silly. It would make a lot more sense to do it the other way round. – Aran-Fey Jun 02 '18 at 14:18
  • This situation can only be produced by using `object` as argument explicitly, which is rarely the case in practice. So I agree with your "asking silly question" viewpoint. But I am not agree with the "undefined behavior" viewpoint, since at least it is allowed (no document says `type` cannot be `object`) and not even an error (no exception thrown). So it is well-defined. – cgsdfc Jun 02 '18 at 15:22