72

I'd like to use a Mixin to always add some init functionality to my child classes which each inherit from different API base classes. Specifically, I'd like to make multiple different child classes that inherit from one of these different API-supplied base classes and the one Mixin, which will always have the Mixin initialization code executed in the same way, without code replication. However, it seems that the __init__ function of the Mixin class never gets called unless I explicitly call it in the Child class's __init__ function, which is less than ideal. I've built up a simple test case:

class APIBaseClassOne(object):
    def __init__(self, *args, **kwargs):
        print (" base ")

class SomeMixin(object):
    def __init__(self, *args, **kwargs):
        print (" mixin before ")
        super(SomeMixin, self).__init__(*args, **kwargs)
        print (" mixin after ")

class MyClass(APIBaseClassOne):
    pass

class MixedClass(MyClass, SomeMixin):
    pass

As you can see in the following output, the Mixin function's init never gets called:

>>> import test
>>> test.MixedClass()
 base
<test.MixedClass object at 0x1004cc850>

Is there a way to do this (have an init function in a Mixin get called) without writing every child class to explicitly invoke the Mixin's init function? (i.e., without having to do something like this in every class:)

class MixedClass(MyClass, SomeMixin):
    def __init__(*args, **kwargs):
        SomeMixin.__init__(self, *args, **kwargs)
        MyClass.__init__(self, *args, **kwargs) 

Btw, if all my child classes were inheriting from same base class, I realize I could create a new middle class that inherits from the base class and the mixin and keep it DRY that way. However, they inherit from different base classes with common functionality. (Django Field classes, to be precise).

martineau
  • 119,623
  • 25
  • 170
  • 301
B Robster
  • 40,605
  • 21
  • 89
  • 122
  • 1
    In general, using multiple inheritance with base classes that weren't designed for it is a bad idea. Mix-in classes are usually designed together, and mixing-in arbitrary classes produces such messes. In any case, if both base classes each have an `__init__` method, how should the interpreter know which one to call, or in which order to call them? – André Caron May 23 '11 at 15:44
  • 2
    @André Caron: It could determine the order like C++ does, where base classes are initialized in declaration order. – martineau May 23 '11 at 16:05

4 Answers4

61

Sorry I saw this so late, but

class MixedClass2(SomeMixin, MyClass):
    pass

>>> m = MixedClass2()
 mixin before 
 base 
 mixin after

The pattern @Ignacio is talking about is called cooperative multiple inheritance, and it's great. But if a base class isn't interested in cooperating, make it the second base, and your mixin the first. The mixin's __init__() (and anything else it defines) will be checked before the base class, following Python's MRO.

This should solve the general question, though I'm not sure it handles your specific use. Base classes with custom metaclasses (like Django models) or with strange decorators (like @martineau's answer ;) can do crazy things.

Matt Luongo
  • 14,371
  • 6
  • 53
  • 64
  • 2
    Even if it was late, this boils it down nicely (although the other answers provide some really good teaching)-- ultimately, an understanding of Python MRO seems critical before embarking on any multiple-inheritance adventures, or unexpected things will happen. THanks! – B Robster May 02 '12 at 18:30
  • Welcome! Absolutely right, it's unfortunate the the MRO is presented as an advanced topic :/ – Matt Luongo May 03 '12 at 02:17
34

Have the base class invoke super().__init__() even though it is a subclass of object. That way all the __init__() methods will be run.

class BaseClassOne(object):
    def __init__(self, *args, **kwargs):
        super(BaseClassOne, self).__init__(*args, **kwargs)
        print (" base ")
Ignacio Vazquez-Abrams
  • 776,304
  • 153
  • 1,341
  • 1,358
  • Thanks! I didn't realize that calling super would cause all __init__() methods to be run. Unfortunately BaseClassOne is an API (Django) supplied base class (i've updated my question to reflect this) but this may get me started in the right direction. – B Robster May 23 '11 at 15:18
  • 11
    It won't cause *all* of them to be run, but it will cause the *next* one to be run, even if it's in a mixin. – Ignacio Vazquez-Abrams May 23 '11 at 15:19
  • Learn something new every day ... ;-) Thanks. – Santa May 23 '11 at 16:44
  • In a multiple-inheritance situation, all classes should use `super()` to find the next class to initialize. – kindall May 23 '11 at 16:45
  • 1
    @kindall: I think many have missed the point that the OP can't change `BaseClassOne` because it part of an API they have no control over. – martineau May 23 '11 at 17:24
  • 2
    @martineau: That fact was revealed only after I posted this answer. And that doesn't make this answer incorrect, just much less relevant for the situation. – Ignacio Vazquez-Abrams May 23 '11 at 17:25
  • Even if you can't change one of the clasess, you could potentially subclass it and give the subclass the desired behavior, or use a wrapper class for the same purpose. Of course, if you have some code you can't change that's a stickler for a particular type, that may not work so well... – kindall May 23 '11 at 17:40
  • @kindall: I tried adding an `__init__` with a `super(...).__init__` call to **both** the OP's `MyClass` and `MixedClass` but the one for `SomeMixin` is still not called when I instantiate a `MixedClass` object -- so perhaps I'm misunderstanding what you're suggesting. – martineau May 23 '11 at 20:01
  • I think for multiple inheritances, you should call all __init__ of parents that you want. FG: `Child(Mixin1, Mixin2):`, if you want to call both parent's __init__ you should call them. (`super(Mixin1, self).__init__()` and `super(Mixin2,self).__init__()`) otherwise, python will call the first one (Mixin1) – Phoenix Jun 13 '19 at 10:38
  • I think only this is CORRECT answer here. I will make separate answer to explain it. – mirek Jun 08 '22 at 09:42
26

Python performs no implicit calls to the __init__ methods of a class' super-class(es)—but it's possible to make it happen automatically. One way is by defining a metaclass for your mixed class(es) that creates or extends the mixed class' __init__ method so that it calls all the listed bases' __init__ functions in the order they were listed.

A second way is to do it is to use a class decorator—which is shown in the Edit section below.

Using a metaclass:

class APIBaseClassOne(object):  # API class (can't be changed)
    def __init__(self, *args, **kwargs):
        print('  APIBaseClassOne.__init__()')

class SomeMixin(object):
    def __init__(self, *args, **kwargs):
        print('  SomeMixin.__init__()')

class MixedClassMeta(type):
    def __new__(cls, name, bases, classdict):
        classinit = classdict.get('__init__')  # Possibly None.

        # Define an __init__ function for the new class.
        def __init__(self, *args, **kwargs):
            # Call the __init__ functions of all the bases.
            for base in type(self).__bases__:
                base.__init__(self, *args, **kwargs)
            # Also call any __init__ function that was in the new class.
            if classinit:
                classinit(self, *args, **kwargs)

        # Add the local function to the new class.
        classdict['__init__'] = __init__
        return type.__new__(cls, name, bases, classdict)

class MixedClass(APIBaseClassOne, SomeMixin):
    __metaclass__ = MixedClassMeta  # important
    # If exists, called after the __init__'s of all the direct bases.
    def __init__(self, *args, **kwargs):
        print('  MixedClass.__init__()')

print('MixedClass():')
MixedClass()

Output:

MixedClass():
  APIBaseClassOne.__init__()
  SomeMixin.__init__()
  MixedClass.__init__()

Edit

Here's how to accomplish the same thing with a class decorator (requires Python 2.6+):

class APIBaseClassOne(object):  # API class (can't be changed)
    def __init__(self, *args, **kwargs):
        print('  APIBaseClassOne.__init__()')

class SomeMixin(object):
    def __init__(self, *args, **kwargs):
        print('  SomeMixin.__init__()')

def mixedomatic(cls):
    """ Mixed-in class decorator. """
    classinit = cls.__dict__.get('__init__')  # Possibly None.

    # Define an __init__ function for the class.
    def __init__(self, *args, **kwargs):
        # Call the __init__ functions of all the bases.
        for base in cls.__bases__:
            base.__init__(self, *args, **kwargs)
        # Also call any __init__ function that was in the class.
        if classinit:
            classinit(self, *args, **kwargs)

    # Make the local function the class's __init__.
    setattr(cls, '__init__', __init__)
    return cls

@mixedomatic
class MixedClass(APIBaseClassOne, SomeMixin):
    # If exists, called after the __init__'s of all the direct base classes.
    def __init__(self, *args, **kwargs):
        print('  MixedClass.__init__()')

print('MixedClass():')
MixedClass()

Notes

For Python < 2.6, use MixedClass = mixedomatic(MixedClass) following the class definition.

In Python 3 the syntax for specifying metaclasses is different, so instead of the:

class MixedClass(APIBaseClassOne, SomeMixin):
    __metaclass__ = MixedClassMeta  # important

shown above, you would need to use:

class MixedClass(APIBaseClassOne, SomeMixin, metaclass=MixedClassMeta):

The class decorator version will work as-is in both versions.

martineau
  • 119,623
  • 25
  • 170
  • 301
  • 1
    Thanks for demonstrating this. While the discussion in the other answer has made me rethink whether its a good idea to begin with, this implementation works and I've learned a lot. – B Robster May 25 '11 at 04:16
  • 1
    @Ben: Yes, mixins are somewhat controversial, and even considered harmful by some, plus using them in Python is somewhat complicated by the fact that it doesn't call base class constructors by default (unlike, say, C++ would). However their use *may* be justified in a situation like yours where you can't modify the API's class -- but you might want to think more about alternatives to do what you need like subclassing or wrapping the API's class instead. Another possibility might be to make an instance of it an attribute of one of your own classes. – martineau May 25 '11 at 14:48
  • not sure, wish i could accept both, but yours has been re-accepted :) – B Robster Dec 03 '11 at 22:46
  • @Ben: Thanks. I can understand your dilemma, but my [biased] opinion is that mine is better because not only does it actually solve your particular problem, it manages to do so in a generic way -- and is thus a little more complicated than just calling the base class's `init()`. Real world problems can be like that. – martineau Dec 04 '11 at 01:02
  • I really like the mixedomatic decorator. I modified it slightly: `classinit = cls.__init__ if '__init__' in cls.__dict__ else None` – Al Conrad Mar 30 '17 at 04:05
  • @AlConrad: You don't say what the rationale is for that change—in other words, why you think it's an improved over `classinit = getattr(cls, '__init__', None)`? – martineau Oct 15 '17 at 15:01
  • @martineau I don't remember. Was the `classinit = getattr(cls, '__init__', None)` always there or was it an edit? I think maybe it used to be just `classinit = cls.__init__` so I suggested the modification, but using getattr is probably equivalent. I use it here: [_mixin_common.py](https://github.com/avolkov1/keras_experiments/blob/master/keras_exp/_mixin_common.py) and here [optimizers.py](https://github.com/avolkov1/keras_experiments/blob/master/keras_exp/multigpu/optimizers.py) – Al Conrad Oct 15 '17 at 16:58
  • @martineau Actually, now that I think about it I had the following problem: ``` @mixedomatic class SomeCls(SomeMixin, SomeBase): pass ``` `SomeCls` doesn't have an `__init__`, but using `getattr` would produce an init b/c SomeMixin and/or SomeBase might have an init. My suggestion would correctly identify that SomeCls does not have an init. I use that in [optimizers.py](https://github.com/avolkov1/keras_experiments/blob/master/keras_exp/multigpu/optimizers.py) I mentioned above. – Al Conrad Oct 15 '17 at 17:13
  • @AlConrad: I'm not sure you understand what the code is doing. First of all, `classinit = getattr(cls, '__init__', None)` produces exactly the same result as what your `if` statement does, but more succinctly. Secondly, the `mixedomatic()` decorator _always_ creates an `__init__()` for the decorated class and the `classinit` variable is used to determine whether the created method needs to call any `__init__()` that might have existed already when it gets called (following calls to the ones of all the direct base classes). – martineau Oct 16 '17 at 16:50
  • @martineau I posted a gist to illustrate. [https://gist.github.com/avolkov1/65032f5ec308b05f1bdb0a20814da7f2](https://gist.github.com/avolkov1/65032f5ec308b05f1bdb0a20814da7f2) Look at lines 56 and 101. See the difference. In the first case you are calling `SomeMixin.__init__` twice. That's a bug. – Al Conrad Oct 16 '17 at 17:23
  • @AlConrad: OK, now I see what you mean. In fact `getattr()` doesn't return the same thing as the `if`-`in` statement in the case of a class decorator, since those are executed after a initial unmodified version of the class has already been created—and `getattr()` follows the MRO of that temporary class and returns the `__init__` in the first match in the inheritance hierarchy (the first base class' `__init__()` here)... – martineau Oct 17 '17 at 01:15
  • ...cont This is different from how metaclasses operate because they're a part of single class construction process that occurs using them, so they don't encounter the issue. Thanks for pointing the decorator problem out and suggesting the fix. – martineau Oct 17 '17 at 01:16
  • Thank you. It is great tutorial and new and important things for me. But lets note that the reason of problems in the question is only the missing super() call in APIBaseClassOne. – mirek Jun 08 '22 at 10:11
0

We can learn much from all answers. But I think only Ignacio's answers is CORRECT here.

In other words and very simply said:

If you rewrite some method, always add super() call inside !!! Any method without super() call will break the run of such method for classes/mixins which follows !!!

So the Ignacio's answer is the proper one for the case if WE create the classes.

Problem is that the 3RD PARTY authors write classes which are buggy in this manner. I have found this stackoverflow question while I was working with Django Rest Framework.

class FinalViewSet(viewsets.ModelViewSet, MyMixin)

will fail (the __init__ from MyMixin will not run), while

class FinalViewSet(MyMixin, viewsets.ModelViewSet)

will make things correct.

So lets go to sources of viewsets.ModelViewSet ...

... and we will find that django.views.generic.base.View has incorrect __init__ method.

So the problems is not at DRF authors but at Django authors, in this case.

mirek
  • 1,140
  • 11
  • 10