6

So I have this metaclass that I want to use for automatic registration of new components, i.e. subclasses of some base component class. When registering a new component, its instance is expected to be passed to the register_component() function that handles that.

Metaclass code (stripped-down version):

class AutoRegisteredMeta(type):
    def __new__(metacls, name, bases, attrs):

        # ... (omitted) check if "name" has already been registered ...

        new_class = super().__new__(metacls, name, bases, attrs)
        register_component(name, new_class())  # RuntimeError(super(): empty __class__ cell)
        return new_class

The problem is that invoking new_class() results in an error - but not for all classes. After some experimenting I realized that this only happens if a subclass calls super().__init__() in its own __init__() method.

Sample component and the base class:

class BaseComponent(metaclass=AutoRegisteredMeta):
    def __init__(self):
        # do some work here ...

class ComponentFoo(BaseComponent):
    def __init__(self):
        super().__init__()  # <--- RuntimeError occurs here
        self.foo = 'bar'

What am I doing wrong here? Reading this I found out that I probably shouldn't be doing instantiation in metaclass'es __new__() or __init__(), right? Can this perhaps be circumvented somehow?

Also, some explanation in layman's terms would be nice, I don't know much of the internals of the CPython implementation.

Thanks in advance!

(FWIW, I use Python 3.3.6, Ubuntu)

EDIT: I'm adding the minimal example that was requested, you can run it directly and see the error in action yourself.

#!/usr/bin/env python3


class AutoRegisteredMeta(type):
    def __new__(metacls, name, bases, attrs):
        new_class = super().__new__(metacls, name, bases, attrs)
        new_class()  # <--- RuntimeError can occur here
        return new_class


class BaseComponent(metaclass=AutoRegisteredMeta):
    def __init__(self):
        print("BaseComponent __init__()")


class GoodComponent(BaseComponent):
    def __init__(self):
        print("GoodComponent __init__()")


class BadComponent(BaseComponent):
    def __init__(self):
        print("BadComponent __init__()")
        super().__init__()  # <--- RuntimeError occurs because of this
Community
  • 1
  • 1
plamut
  • 3,085
  • 10
  • 29
  • 40
  • 2
    Your registration code actually instantiates an instance of the class before the metaclass has finished. `register_component(name, new_class)` might be better. – Charlie Clark Jun 23 '15 at 12:50
  • Could you reorganise this into a [minimal example](http://stackoverflow.com/help/mcve) that actually runs (include `register_component`, `BaseComponent.__init__`, ...)? – jonrsharpe Jun 23 '15 at 12:58
  • @CharlieClark Yes, I know, that's exactly the problem. This made me thinking, though, that registering the class itself rather than its instance would be better, just what you, too, suggested in your comment. Registering a component instance is in line with what's currently done in the rest of the system, but now I have a good argument to refactor that. :) – plamut Jun 23 '15 at 12:59
  • @jonrsharpe I've also tried that, but same error. It's explained in the discussion I linked. If I understood correctly, metaclass must first finish with its class creation process before this class can be instantiated (if that class uses super(), that is). – plamut Jun 23 '15 at 13:02
  • @plamut tried what? I'm not suggesting a way to fix it, just asking you to provide the rest of us a way to recreate the issue. – jonrsharpe Jun 23 '15 at 13:04
  • @jonrsharpe Sorry, never mind, I read too fast. I've added the minimal example you requested that produces the error. – plamut Jun 23 '15 at 13:32
  • 1
    https://mail.python.org/pipermail/python-ideas/2015-February/032136.html – Ashwini Chaudhary Jun 23 '15 at 13:41
  • *"When registering a new component, its instance is expected to be passed to the register_component() function"* - why an instance, not the class? – jonrsharpe Jun 23 '15 at 13:45
  • It's the way the existing system works (I did not design that). That said, I'll change the registration mechanism to register classes instead, and only instantiate them later when they are actually needed (mentioned in the first couple of comments). @AshwiniChaudhary Thanks for the link! I read the discussion and it seems that it is indeed not possible do what I initially wanted, not until [PEP 487](https://www.python.org/dev/peps/pep-0487/) is implemented in Python3.5, right? That `__init_subclass__` hook really looks useful. – plamut Jun 23 '15 at 14:11
  • 1
    You might also look at some kind of class decorator / scan system for registration. And think about why those approaches are preferred over your code. – Charlie Clark Jun 23 '15 at 14:22
  • @CharlieClark Decorators were my first thought, but since they would have to be applied to every new class, they wouldn't provide any significant advantage over the manual calls to `register_component()` after each class definition, thus I discarded the idea. I like the scanner idea, though. Nevertheless, I eventually opted for registering the classes themselves, as suggested, and instantiating them later when needed. It required the least amount of changes. Thanks everyone for your input! – plamut Jun 24 '15 at 09:04

1 Answers1

4

maybe this works:

#!/usr/bin/env python3


class AutoRegisteredMeta(type):
    def __new__(metacls, name, bases, attrs):
        new_class = super().__new__(metacls, name, bases, attrs)
        new_class()
        return new_class


class BaseComponent(metaclass=AutoRegisteredMeta):
    def __init__(self):
        print("BaseComponent __init__()")


class GoodComponent(BaseComponent):
    def __init__(self):
        print("GoodComponent __init__()")


class BadComponent(BaseComponent):
    def __init__(self):
        print("BadComponent __init__()")
        super(self.__class__, self).__init__()
svs
  • 431
  • 1
  • 7
  • 20
  • I've tried and it indeed works, thanks. Adding explicit arguments to `super()`, i.e. `self.__class__` and `self`, made the problem disappear. Accepting this answer. BTW, you might want to remove the last comment in the source as it does not apply anymore. – plamut Jul 22 '15 at 22:01
  • I've just researched it – svs Aug 09 '15 at 14:39
  • BTW, do not use `super(self.__class__, self).__init__()` as it can result in infinite loops - `self` might be an instance of a subclass, and using `super()` like this can actually invoke the `__init__()` method that contains that `super()` statement, over and over again. – plamut Nov 27 '17 at 19:29