python super() - two argument version in context of __new__

Question

I've read the documentation for super() multiple times but I still don't get what the two argument version returns.

Return a proxy object that delegates method calls to a parent or sibling class of type. This is useful for accessing inherited methods that have been overridden in a class.

1. What is a proxy object?
2. parent or sibling?

If the second argument is an object, isinstance(obj, type) must be true. If the second argument is a type, issubclass(type2, type) must be true (this is useful for classmethods).

3. That's all nice but what is returned? And what does the syntax super(a,b) mean?

In context of __new__,

Typical implementations create a new instance of the class by invoking the superclass’s __new__() method using super(currentclass, cls).__new__(cls[, ...]) with appropriate arguments and then modifying the newly-created instance as necessary before returning it.

super(currentclass, cls).__new__(cls[, ...])

If currentclass is equal to cls, you have super(currentclass, currentclass)

4. What does that return? What does that syntax mean?
5. How does this statement return an instance of cls so that __init__ will be called on the returned object?

Answer 1

The answer to "what does super return" isn't anything like "it returns a list" or "it returns a modified copy of the second object". super returns an object of type super, a type designed specifically to have the qualities the documentation says it has.

Perhaps it would help to show a pure-Python implementation of super. If super wasn't written in C, it would look basically like this:

class super(object):
    def __init__(self, klass, object_or_klass):
        # The real super can be called with 0 arguments on Python 3,
        # but digging into the magic that makes that work isn't relevant here.

        if isinstance(object_or_klass, klass):
            mro = type(object_or_klass).__mro__
            self.obj_type = type(object_or_klass)
            self.obj = object_or_klass
        elif issubclass(object_or_klass, klass):
            mro = object_or_klass.__mro__
            self.obj_type = object_or_klass
            self.obj = None
        else:
            raise TypeError

        # Set up a copy of the MRO to search,
        # with everything up to and including klass skipped
        self.searchlist = mro[mro.index(klass)+1:]

    def __getattribute__(self, name):
        # self.searchlist would be infinite recursion, as would super().__getattribute__
        searchlist = object.__getattribute__(self, 'searchlist')

        # Search the method resolution order for the attribute we want.
        for klass in searchlist:
            if name in klass.__dict__:
                attr = klass.__dict__[name]
                break
        else:
            raise AttributeError

        if hasattr(attr, '__get__'):
            # Handle descriptors.
            obj = object.__getattribute__(self, 'obj')
            obj_type = object.__getattribute__(self, 'obj_type')
            attr = attr.__get__(obj, obj_type)
        return attr

Now you can see that super(a, b) constructs a super object, and super(a, b).whatever calls the __getattribute__ method of that super object to search the method resolution order of the second argument for the attribute we want. The attribute lookup procedure for b.whatever is very similar, just without chopping off the first part of the MRO, and checking the instance dict if b isn't a class.

Answer 2

You should definitely watch Raymond Hettinger's talk from PyCon 2015, Super is considered super!

But if not, why not add a bunch of print statements to answer your questions?

# (object) is only required in Python2
class Mom(object):
    def __init__(self, *args, **kwargs):
        print('Mom is initializing - args: {!r} kwargs: {!r}'.format(
              args, kwargs))

    def do_something(self):
        print('Doing some Mom thing')


class Dad(object):
    def __init__(self, *args, **kwargs):
        print('Dad is initializing - args: {!r} kwargs: {!r}'.format(
              args, kwargs))

    def do_something(self):
        print('Doing some Dad thing')


class Sister(Mom, Dad):
    def __init__(self, name):
        print('initializing a Sister with name: {!r}'.format(name))
        parent = super(Sister, self)
        print(type(parent))
        print(parent)
        print('Calling super __init__')
        parent.__init__(name)

    def do_something(self, value):
        if value == 5:
            print('calling method on super')
            super(Sister, self).do_something()
        else:
            print('Sister did something')


class Brother(Mom):
    def __init__(self, name):
        print('initializing a Brother with name: {!r}'.format(name))
        parent = super(Brother, self)
        print(type(parent))
        print(parent)
        print('Calling super __init__')
        parent.__init__(name)

    def do_something(self, value):
        if value == 5:
            print('calling method on super')
            super(Brother, self).do_something()
        else:
            print('Brother did something')


b = Brother('Bear')
s = Sister('Moon')

b.do_something(3)
b.do_something(5)

s.do_something(3)
s.do_something(5)

That produces the following output (with added commentary):

<type 'super'>
<super: <class 'Brother'>, <Brother object>>
Calling super __init__
Mom is initializing - args: ('Bear',) kwargs: {}

Apparently, super returns a class of type super. Which, according to the documentation, is a proxy object. A proxy, by definition, is a replacement for something else. In this case, the proxy is a replacement for getting Mom. You can see that when we actually call the __init__ function, that Mom's init function is called.

initializing a Sister with name: 'Moon'
<type 'super'>
<super: <class 'Sister'>, <Sister object>>
Calling super __init__
Mom is initializing - args: ('Moon',) kwargs: {}

You'll notice here, that Dad's init function isn't called. That's because, if you watch Raymond's talk, you'll know that super looks for the function on parents from left to right.

Brother did something
calling method on super
Doing some Mom thing

You see the same behavior repeated here

Sister did something
calling method on super
Doing some Mom thing

If you change the order on Sister, to Dad, Mom, you'll see that those calls change:

initializing a Brother with name: 'Bear'
<type 'super'>
<super: <class 'Brother'>, <Brother object>>
Calling super __init__
Mom is initializing - args: ('Bear',) kwargs: {}
initializing a Sister with name: 'Moon'
<type 'super'>
<super: <class 'Sister'>, <Sister object>>
Calling super __init__
Dad is initializing - args: ('Moon',) kwargs: {}
Brother did something
calling method on super
Doing some Mom thing
Sister did something
calling method on super
Doing some Dad thing

To sum up:

1. A proxy is something that stands in for something else. In our case, super is a proxy for both Mom & Dad, depending on which order they're inherited from.

2. Honestly, I couldn't get any kind of meaningful sibling thing to work. I'm not sure when you'd even need that anyway.

3. An instance of super is returned.

4. That would really only work if you had something that inherited from type, e.g. super(type, type), because you need an instance of that type. object would also work because:

   print(isinstance(object, object)) print(isinstance(type, type)) print(isinstance(object, type)) print(isinstance(type, object))

There's some pretty involved magic going on around that process.

5. Are you asking how functions return things? Because new is a function - and it returns or it doesn't. object.__new__ returns something. I'm not sure where the actual source code is for that function, or I could point you there if you were interested in the exact mechanism.