The metaclass's "__init_subclass__" method doesn't work in the class constructed by this metaclass

Question

My question was inspired by this question.

The problem there is the 3 level class model - the terminating classes (3-rd level) only should be stored in the registry, but the 2-nd level are interfering and also have stored, because they are subclasses of 1-st level.

I wanted to get rid of 1-st level class by using metaclass. By this way the only 2 class levels are left - base classes for each group of settings and their childs - various setting classes, inherited from the according base class. The metaclass serves as a class factory - it should create base classes with needed methods and shouldn't be displayed in the inheritance tree.

But my idea doesn't work, because it seems that the __init_subclass__ method (the link to method) doesn't copied from the metaclass to constructed classes. In contrast of __init__ method, that works as I were expected.

Code snippet № 1. The basic framework of the model:

class Meta_Parent(type):
    pass

class Parent_One(metaclass=Meta_Parent):
    pass

class Child_A(Parent_One):
    pass

class Child_B(Parent_One):
    pass

class Child_C(Parent_One):
    pass

print(Parent_One.__subclasses__())

Output:

[<class '__main__.Child_A'>, <class '__main__.Child_B'>, <class '__main__.Child_C'>]

I have wanted to add functionality to the subclassing process of the above model, so I have redefined the type's builtin __init_subclass__ like this:

Code snippet № 2.

class Meta_Parent(type):
    def __init_subclass__(cls, **kwargs):
        super().__init_subclass__(**kwargs)
        print(cls)

From my point of view, now every new class, constructed by Meta_Parent metaclass (for example, Parent_One) should have __init_subclass__ method and thus, should print the subclass name when every class is inherited from this new class, but it prints nothing. That is, my __init_subclass__ method doesn't called when inheritance happens.

It works if Meta_Parent metaclass is directly inherited though:

Code snippet № 3.

class Meta_Parent(type):
    def __init_subclass__(cls, **kwargs):
        super().__init_subclass__(**kwargs)
        print(cls)

class Child_A(Meta_Parent):
    pass

class Child_B(Meta_Parent):
    pass

class Child_C(Meta_Parent):
    pass

Output:

<class '__main__.Child_A'>
<class '__main__.Child_B'>
<class '__main__.Child_C'>

Nothing strange here, the __init_subclass__ was created exactly for this purpose.

I were thinking at a moment, that dunder methods are belong to metaclass only and are not passed to new constructed classes, but then, I try the __init__ method and it works as I were expecting in the beginning - looks like the link to __init__ have copied to every metaclass's class.

Code snippet № 4.

class Meta_Parent(type):
    def __init__(cls, name, base, dct):
        super().__init__(name, base, dct)
        print(cls)

Output:

<class '__main__.Parent_One'>
<class '__main__.Child_A'>
<class '__main__.Child_B'>
<class '__main__.Child_C'>

The questions:

Why __init__ works, but __init_subclass__ doesn't?
Is it possible to implement my idea by using metaclass?

score 5 · Accepted Answer · edited Jun 20 '20 at 09:12

1. Why `init` works, but `__init_subclass__` doesn't?

I found the answer by debugging CPython by GDB.

The creation of a new class (type) starts in the type_call() function. It does two main things: a new type object creation and this object initialization.
obj = type->tp_new(type, args, kwds); is an object creation. It calls the type's tp_new slot with passed arguments. By default the tp_new stores reference to the basic type object's tp_new slot, but if any ancestor class implements the __new__ method, the reference is changing to the slot_tp_new dispatcher function. Then the type->tp_new(type, args, kwds); callsslot_tp_new function and it, in own turn, invokes the search of __new__ method in the mro chain. The same happens with tp_init.
The subclass initialization happens at the end of new type creation - init_subclass(type, kwds). It searches the __init_subclass__ method in the mro chain of the just created new object by using the super object. In my case the object's mro chain has two items:
```
print(Parent_One.__mro__)
### Output
(<class '__main__.Parent_One'>, <class 'object'>).
```
int res = type->tp_init(obj, args, kwds); is an object initialization. It also searches the __init__ method in the mro chain, but use the metaclass mro, not the just created new object's mro. In my case the metaclass mro has three item:
```
print(Meta_Parent.__mro__)
###Output
(<class '__main__.Meta_Parent'>, <class 'type'>, <class 'object'>)
```

The simplified execution diagram:

So, the answer is: __init_subclass__ and __init__ methods are searched in the different places:

the __init_subclass__ firstly is searched in the Parent_One's __dict__, then in the object's __dict__.
the __init__ is searched in this order: Meta_Parent's __dict__, type's __dict__, object's __dict__.

2. Is it possible to implement my idea by using metaclass?

I came up with following solution. It has drawback - the __init__ method is called by each subclass, the children included, that means - all subclasses have registry and __init_subclass__ attributes, which is needless. But it works as I were requesting in the question.

#!/usr/bin/python3

class Meta_Parent(type):
    def __init__(cls, name, base, dct, **kwargs):
        super().__init__(name, base, dct)
        # Add the registry attribute to the each new child class.
        # It is not needed in the terminal children though.
        cls.registry = {}
        
        @classmethod
        def __init_subclass__(cls, setting=None, **kwargs):
            super().__init_subclass__(**kwargs)
            cls.registry[setting] = cls

        # Assign the nested classmethod to the "__init_subclass__" attribute
        # of each child class.
        # It isn't needed in the terminal children too.
        # May be there is a way to avoid adding these needless attributes
        # (registry, __init_subclass__) to there. I don't think about it yet.
        cls.__init_subclass__ = __init_subclass__

# Create two base classes.
# All child subclasses will be inherited from them.
class Parent_One(metaclass=Meta_Parent):
    pass

class Parent_Two(metaclass=Meta_Parent):
    pass

### Parent_One's childs
class Child_A(Parent_One, setting='Child_A'):
    pass

class Child_B(Parent_One, setting='Child_B'):
    pass

class Child_C(Parent_One, setting='Child_C'):
    pass

### Parent_Two's childs
class Child_E(Parent_Two, setting='Child_E'):
    pass

class Child_D(Parent_Two, setting='Child_D'):
    pass

# Print results.
print("Parent_One.registry: ", Parent_One.registry)
print("#" * 100, "\n")
print("Parent_Two.registry: ", Parent_Two.registry)

Output

Parent_One.registry:  {'Child_A': <class '__main__.Child_A'>, 'Child_B': <class '__main__.Child_B'>, 'Child_C': <class '__main__.Child_C'>}
#################################################################################################### 

Parent_Two.registry:  {'Child_E': <class '__main__.Child_E'>, 'Child_D': <class '__main__.Child_D'>}

score 1 · Answer 2 · answered Dec 06 '20 at 23:48

The solution I came up with and use/like is:

class Meta_Parent(type):
    def _init_subclass_override(cls, **kwargs):
        super().__init_subclass__(**kwargs)
        # Do whatever... I raise an exception if something is wrong
        #
        # i.e
        # if sub-class's name does not start with "Child_"
        #     raise NameError
        #
        # cls is the actual class, Child_A in this case

class Parent_One(metaclass=Meta_Parent):
    @classmethod
    def __init_subclass__(cls, **kwargs):
        Meta_Parent._init_subclass_override(cls, **kwargs)


### Parent_One's childs
class Child_A(Parent_One):
    pass

I like this because it DRYs the sub-class creation code/checks. At the same time, if you see Parent_One, you know that there is something happening whenever a sub-class is created.

I did it while mucking around to mimic my own Interface functionality (instead of using ABC), and the override method checks for existence of certain methods in the sub-classes.

One can argue whether the override method really belongs in the metaclass, or somewhere else.

The metaclass's "__init_subclass__" method doesn't work in the class constructed by this metaclass

2 Answers2

1. Why __init__ works, but __init_subclass__ doesn't?

2. Is it possible to implement my idea by using metaclass?

1. Why `init` works, but `__init_subclass__` doesn't?