Auto instantiation of class after import in Python

Question

I want to automatically instantiate some classes in Python just after their modules have been imported, so there is no need to direct instantiation call. My setup can be stripped down to following code:

class MetaTest(type):

    _instances = {}

    def __init__(cls, name, bases, attrs):
        super(MetaTest, cls).__init__(name, bases, attrs)
        MetaTest._instances[name] = cls()


class Foo(object):

    __metaclass__ = MetaTest

    def __init__(self):
        super(Foo, self).__init__()

class Bar(Foo):

    def __init__(self):
        super(Bar, self).__init__()

But the instantiation fails on the super with:

    super(Foo, self).__init__()
NameError: global name 'Foo' is not defined

By any chance, could there be the way how to do this?

Answer 1

First note:

for this sort of code, it is easier to be using Python 3. More about this on the first solution and at the end of the answer.

So, the name of the class itself Foo won't, obviously, be available inside the body of class Foo: this name is only bound after the class instantiation is completely performed.

In Python 3, there is the parameterless version of super() call, which can direct one to the superclass with no need for a explicit reference to the class itself. It is one of the few exceptions the language make to its very predictable mechanisms. It binds data to the super call at compile time. However, even in Python 3 it is not possible to call a method that makes use of super while the class is still being istantiated - i.e. before returning from the metaclass __new__ and __init__ methods. The data needed by the parameterless super is silently bound after that.

However, in Python 3.6+ there is a mechanism that even precludes a metaclass: a class method named __init_subclass__ will be called on the superclasses after a class is created, and it could make normal use of (parameterless) super. This would work with Python 3.6:

class Base:

   _instances = {}

   def __init_subclass__(cls):
        __class__._instances[cls.__name__] = cls()

   ...

class Foo(Base):

    def __init__(self):
        super().__init__()

Here, __class__ (not obj.__class__) is another magic name, available along with parameterless super, that always reference the class within which body it is used, never a subclass.

Now, a couple considerations on other methods: no code within the metaclass itself could work to instantiate a class, and the class methods needing to have a reference to the class in its name do not help. A class decorator would also not work, because the name is only bound after the return from the decorator. If you are coding your system within a framework which have an event loop, you could, on the metaclass __init__ schedule an event to create a class instance - that would work, but you need to be coding within such a context (gobject, Qt, django, sqlalchemy, zope/pyramid, Python asyncIO/tulip are examples of frameworks which have an event system that could make the needed callbacks)

Now, if you really need this in Python 2, I think the easiest way is to create a function to be called at the footer of each of your modules that would "register" the recently created classes. Something along:

class MetaTest(type):

    _instances = {}

def register():
    import sys
    module_namespace = sys._getframe(1).f_globals

    for name, obj in f_globals.items():
        if isinstance(obj, MetaTest):
            MetaTest._instances[obj.__name__] = obj()

again: a a call to this "register" function should come at the last line of each of your modules.

The "meta" thing in here is introspecting the frame of the caller function itself to get its global variables automatically. You can get rid with it by making globals() a required explicit parameter to the register function, and keeping the code easier to understand by third-parties.

Why Python 3

As of today, Python 2 is 2 years from End of Line. And a lot of improvements have been happening in the language in the last 8 years, including features in the class and metaclass models. I've seem a lot of people seems to stick with python 2 because as "python" is linked to Python 2 in their Linux installs, they think that is the default. Not true: it is just there for backwards compatibility - more and more Linux install use python3 as the default Python, usually installed alongside Python 2. And for your projects, it is nicer to use an isolated environment created with virtuaelnv anyway.

Answer 2

Solution 1: Modify sys.modules

This is a Python 2 version. Adding the class to its module works, because it solves the problem that the class does not exits yet when you use super():

import sys

class MetaTest(type):

    _instances = {}

    def __init__(cls, name, bases, attrs):
        setattr(sys.modules[cls.__module__], cls.__name__, cls)
        super(MetaTest, cls).__init__(name, bases, attrs)
        MetaTest._instances[name] = cls()


class Foo(object):

    __metaclass__ = MetaTest

    def __init__(self):
        print('init Foo')
        super(Foo, self).__init__()

class Bar(Foo):

    def __init__(self):
        print('init Bar')
        super(Bar, self).__init__()

print(MetaTest._instances)

Output:

init Foo
init Bar
init Foo
{'Foo': <__main__.Foo object at 0x10cef90d0>, 'Bar': <__main__.Bar object at 0x10cef9110>}

Solution 2: Use the future library

The future library offers as super() for Python 2 that works just like the one for Python 3, i.e. without arguments. This helps:

from builtins import super

class MetaTest(type):

    _instances = {}

    def __init__(cls, name, bases, attrs):
        super(MetaTest, cls).__init__(name, bases, attrs)
        MetaTest._instances[name] = cls()


class Foo(object):

    __metaclass__ = MetaTest

    def __init__(self):
        print('init Foo')
        super().__init__()

class Bar(Foo):

    def __init__(self):
        print('init Bar')
        super().__init__()

print(MetaTest._instances)

Output:

init Foo
init Bar
init Foo
{'Foo': <__main__.Foo object at 0x1066b39d0>, 'Bar': <__main__.Bar object at 0x1066b3b50>}

Answer 3

What jsbueno said. I've been trying to make this work in Python 2, but without much success. Here's the closest I got:

class MetaTest(type):
    _instances = {}

    def __init__(cls, name, bases, attrs):
        super(MetaTest, cls).__init__(name, bases, attrs)
        MetaTest._instances[name] = cls()

class Foo(object):
    __metaclass__ = MetaTest

class Bar(Foo):
    pass

for name, obj in Bar._instances.items():
    print name, obj, type(obj)

output

Foo <__main__.Foo object at 0xb74626ec> <class '__main__.Foo'>
Bar <__main__.Bar object at 0xb746272c> <class '__main__.Bar'>

However, if you try to give either Foo or Bar __init__ methods, then it all falls to pieces, for the reasons given by jsbueno.

Here's a Python 3.6 version using the argless form of super that's more successful:

class MetaTest(type):
    _instances = {}

    def __init__(cls, name, bases, attrs):
        print('cls: {}\nname: {}\nbases: {}\nattrs: {}\n'.format(cls, name, bases, attrs))
        super().__init__(name, bases, attrs)
        MetaTest._instances[name] = cls()

class Foo(metaclass=MetaTest):
    def __init__(self):
        super().__init__()

class Bar(Foo):
    def __init__(self):
        super().__init__()

for name, obj in Bar._instances.items():
    print(name, obj, type(obj))

output

cls: <class '__main__.Foo'>
name: Foo
bases: ()
attrs: {'__module__': '__main__', '__qualname__': 'Foo', '__init__': <function Foo.__init__ at 0xb7192dac>, '__classcell__': <cell at 0xb718c2b4: MetaTest object at 0xb718d3cc>}

cls: <class '__main__.Bar'>
name: Bar
bases: (<class '__main__.Foo'>,)
attrs: {'__module__': '__main__', '__qualname__': 'Bar', '__init__': <function Bar.__init__ at 0xb7192d64>, '__classcell__': <cell at 0xb718c2fc: MetaTest object at 0xb718d59c>}

Foo <__main__.Foo object at 0xb712514c> <class '__main__.Foo'>
Bar <__main__.Bar object at 0xb71251ac> <class '__main__.Bar'>

---

I must confess that I'm not very comfortable with this whole concept of classes creating an instance of themself automatically: "Explicit is better than implicit". I think jsbueno's suggestion of using a registration function is a good compromise.