Why does `type.new` call `__init_subclass__`?

Question

I noticed that I couldn't use __init_subclass__ with Django model classes in quite the way I wanted to. It seems that the metaclass hasn't finished creating a child class by the time that a parent class' __init_subclass__ method is run. While I understand what the issue is, and can circumvent it by making a custom metaclass, what I don't understand is why!

In my head, I tend to think that any calls like __new__ should be done before any calls like __init__ happen. But this isn't the case for metaclasses and __init_subclass__, as demonstrated here:

class MetaMeta(type):
    print('parsing MetaMeta')
    def __call__(cls, *args, **kwargs):
        print('entering MetaMeta.__call__')
        instance = super().__call__(*args, **kwargs)
        print('leaving MetaMeta.__call__')
        return instance

class Meta(type, metaclass=MetaMeta):
    print('parsing Meta')
    def __init__(self, *args, **kwargs):
        print('  entering Meta.__init__')
        super().__init__(*args, **kwargs)
        print('  leaving Meta.__init__')
    def __new__(cls, *args, **kwargs):
        print(f'  entering Meta.__new__')
        instance = super().__new__(cls, *args, **kwargs)
        print('  leaving Meta.__new__')
        return instance

class Parent(object, metaclass=Meta):
    print('parsing Parent')
    def __init_subclass__(cls, *args, **kwargs):
        print('    entering Parent.__init_subclass__')
        super().__init_subclass__(*args, **kwargs)
        print('    leaving Parent.__init_subclass__')

class Child(Parent):
    print('parsing Child')

Which results in:

parsing MetaMeta
parsing Meta
parsing Parent
entering MetaMeta.__call__
  entering Meta.__new__
  leaving Meta.__new__
  entering Meta.__init__
  leaving Meta.__init__
leaving MetaMeta.__call__
parsing Child
entering MetaMeta.__call__
  entering Meta.__new__
    entering Parent.__init_subclass__
    leaving Parent.__init_subclass__
  leaving Meta.__new__
  entering Meta.__init__
  leaving Meta.__init__
leaving MetaMeta.__call__

A metaclass can still be setting up the class in Meta.__new__ after __init_subclass__ is called. Which seems odd to me. Why is that the case, and is there any way to provide code in Parent (without a custom metaclass) that is run completely after Meta.__new__ (and probably before Meta.__init__)?

Or am I missing something completely?

FYI, I found some related topics, but not quite what I was looking for:

Perhaps a more concise way to ask this question is "why does Python (v3.9 at least) have Meta.__new__ invoke Parent.__init_subclass__, instead of having MetaMeta.__call__ invoke it immediately after __new__ is complete?

Note that after asking, I did find some python.org discussion around this topic, but I don't think they clarify why:

score 2 · Answer 1 · answered Mar 04 '21 at 21:10

Tricky.

So, it is this way, because it is made so in the language. When adding features for the class creation processes, people did not allow for customization of when __init_subclass__, or the decriptors __set_name__, or the final linearization (mro) is calculated: that is all done at once inside type.__new__ - and any metaclass written will have to call type.__new__ at some point.

However, there is a possible workaround: if type.__new__ won't "see" an __init_subclass__ method, it won't be called.

Therefore, it is possible for a child metaclass to stash away __init_subclass__, call the parent __new__, and then, before leaving its own __new__ to restore and call __init_subclass__.

So, if the problem is specifcially that you need __init_subclass__ to run after Django's metaclass __new__ is completly done, I can think of two options, which both involve inheriting from Django's ORM metaclass, modifying it, and using that as the metaclass for your models.

Then the first option is simply to use another method name than __init_subclass__ in your project. Your custom metaclass calls super().__new__() , Django and Python do their thing, and you call your __init_subclass2__ (or whaterver name you pick). I think this is the most maintainable and straightforward way to go.

The second option would be what I mentioned earlier: your __new__ method checks all bases for any occurrence of __init_subclass__, delete then temporarily from the classes where they are, storing the original methods, call super().__new__(), then restore the __init_subclass__ methods, and call them afterwards. This has the advantage of working with existing __init_subclass__ in the hyerarchy as they are (as long as the classes themselves are written in Python and not native code: in which case temporarily removing the method won't work). It has the severe disadvantage of ou having to scan the mro by yourself (ou will have to re-do the linearization) searching all existing __init_subclass__ and restoring them afterwards - there might be some hard to spot corner cases.

Why does `type.__new__` call `__init_subclass__`?

1 Answers1

Why does `type.new` call `__init_subclass__`?