How does one create a metaclass?

Question

I have a rough idea of what meta-classes are. They are the classes of which class objects are based on (because classes are objects in Python). But could someone explain (with code) how one goes about creating one.

Answer 1

There are (at this point) two key methods in a metaclass:

• __prepare__, and
• __new__

__prepare__ lets you supply a custom mapping (such as an OrderedDict) to be used as the namespace while the class is being created. You must return an instance of whatever namespace you choose. If you don't implement __prepare__ a normal dict is used.

__new__ is responsible for the actual creation/modification of the final class.

A bare-bones, do-nothing-extra metaclass would look like:

class Meta(type):

    def __prepare__(metaclass, cls, bases):
        return dict()

    def __new__(metacls, cls, bases, clsdict):
        return super().__new__(metacls, cls, bases, clsdict)

A simple example:

Say you want some simple validation code to run on your attributes -- like it must always be an int or a str. Without a metaclass, your class would look something like:

class Person:
    weight = ValidateType('weight', int)
    age = ValidateType('age', int)
    name = ValidateType('name', str)

As you can see, you have to repeat the name of the attribute twice. This makes typos possible along with irritating bugs.

A simple metaclass can address that problem:

class Person(metaclass=Validator):
    weight = ValidateType(int)
    age = ValidateType(int)
    name = ValidateType(str)

This is what the metaclass would look like (not using __prepare__ since it is not needed):

class Validator(type):
    def __new__(metacls, cls, bases, clsdict):
        # search clsdict looking for ValidateType descriptors
        for name, attr in clsdict.items():
            if isinstance(attr, ValidateType):
                attr.name = name
                attr.attr = '_' + name
        # create final class and return it
        return super().__new__(metacls, cls, bases, clsdict)

A sample run of:

p = Person()
p.weight = 9
print(p.weight)
p.weight = '9'

produces:

9
Traceback (most recent call last):
  File "simple_meta.py", line 36, in <module>
    p.weight = '9'
  File "simple_meta.py", line 24, in __set__
    (self.name, self.type, value))
TypeError: weight must be of type(s) <class 'int'> (got '9')

---

Notes

This example is simple enough it could have also been accomplished with a class decorator, but presumably an actual metaclass would be doing much more.

In Python 2.x, the __prepare__ method doesn't exist, and the class speficies its metaclass by including a class variable __metaclass__ = ..., like this:

class Person(object):
    __metaclass__ = ValidateType

The 'ValidateType' class for reference:

class ValidateType:
    def __init__(self, type):
        self.name = None  # will be set by metaclass
        self.attr = None  # will be set by metaclass
        self.type = type
    def __get__(self, inst, cls):
        if inst is None:
            return self
        else:
            return inst.__dict__[self.attr]
    def __set__(self, inst, value):
        if not isinstance(value, self.type):
            raise TypeError('%s must be of type(s) %s (got %r)' %
                    (self.name, self.type, value))
        else:
            inst.__dict__[self.attr] = value

Answer 2

I've just written a fully commented example of a metaclass. It's in Python 2.7. I'm sharing it here and hope that it can help you understand more about the __new__, __init__, __call__, __dict__ methods and the concept of bounded/unbounded in Python, as well as the use of metaclasses.

The problem with a metaclass, I feel, is that it has too many places where you can do the same things, or similar yet with some slight differences. So my comments and test cases mainly emphasizes where to write what, what goes to where at certain points, and what are accessible to a certain object.

The example tries to build a class factory while maintaining well-formed class definitions.

from pprint import pprint
from types import DictType

class FactoryMeta(type):
    """ Factory Metaclass """

    # @ Anything "static" (bounded to the classes rather than the instances) 
    #   goes in here. Or use "@classmethod" decorator to bound it to meta. 
    # @ Note that these members won't be visible to instances, you have to 
    #   manually add them to the instances in metaclass' __call__ if you wish
    #   to access them through a instance directly (see below).
    extra = "default extra"
    count = 0

    def clsVar(cls):
        print "Class member 'var': " + str(cls.var)

    @classmethod
    def metaVar(meta):
        print "Metaclass member 'var': " + str(meta.var)

    def __new__(meta, name, bases, dict):
        # @ Metaclass' __new__ serves as a bi-functional slot capable for
        #   initiating the classes as well as alternating the meta.
        # @ Suggestion is putting majority of the class initialization code
        #   in __init__, as you can directly reference to cls there; saving 
        #   here for anything you want to dynamically added to the meta (such 
        #   as shared variables or lazily GC'd temps).
        # @ Any changes here to dict will be visible to the new class and their 
        #   future instances, but won't affect the metaclass. While changes 
        #   directly through meta will be visible to all (unless you override 
        #   it later).
        dict['new_elem'] = "effective"
        meta.var = "Change made to %s by metaclass' __new__" % str(meta)
        meta.count += 1
        print "================================================================"
        print " Metaclass's __new__ (creates class objects)"
        print "----------------------------------------------------------------"
        print "Bounded to object: " + str(meta)
        print "Bounded object's __dict__: "
        pprint(DictType(meta.__dict__), depth = 1)
        print "----------------------------------------------------------------"
        print "Parameter 'name': " + str(name)
        print "Parameter 'bases': " + str(bases)
        print "Parameter 'dict': "
        pprint(dict, depth = 1)
        print "\n"
        return super(FactoryMeta, meta).__new__(meta, name, bases, dict)

    def __init__(cls, name, bases, dict):
        # @ Metaclass' __init__ is the standard slot for class initialization.
        #   Classes' common variables should mainly goes in here.
        # @ Any changes here to dict won't actually affect anything. While 
        #   changes directly through cls will be visible to the created class 
        #   and its future instances. Metaclass remains untouched.
        dict['init_elem'] = "defective"
        cls.var = "Change made to %s by metaclass' __init__" % str(cls)
        print "================================================================"
        print " Metaclass's __init__ (initiates class objects)"
        print "----------------------------------------------------------------"
        print "Bounded to object: " + str(cls)
        print "Bounded object's __dict__: "
        pprint(DictType(cls.__dict__), depth = 1)
        print "----------------------------------------------------------------"
        print "Parameter 'name': " + str(name)
        print "Parameter 'bases': " + str(bases)
        print "Parameter 'dict': "
        pprint(dict, depth = 1)
        print "\n"
        return super(FactoryMeta, cls).__init__(name, bases, dict)

    def __call__(cls, *args):
        # @ Metaclass' __call__ gets called when a class name is used as a 
        #   callable function to create an instance. It is called before the 
        #   class' __new__.
        # @ Instance's initialization code can be put in here, although it 
        #   is bounded to "cls" rather than instance's "self". This provides 
        #   a slot similar to the class' __new__, where cls' members can be 
        #   altered and get copied to the instances.
        # @ Any changes here through cls will be visible to the class and its 
        #   instances. Metaclass remains unchanged.
        cls.var = "Change made to %s by metaclass' __call__" % str(cls)
        # @ "Static" methods defined in the meta which cannot be seen through 
        #   instances by default can be manually assigned with an access point 
        #   here. This is a way to create shared methods between different 
        #   instances of the same metaclass.
        cls.metaVar = FactoryMeta.metaVar
        print "================================================================"
        print " Metaclass's __call__ (initiates instance objects)"
        print "----------------------------------------------------------------"
        print "Bounded to object: " + str(cls)
        print "Bounded object's __dict__: "
        pprint(DictType(cls.__dict__), depth = 1)
        print "\n"
        return super(FactoryMeta, cls).__call__(*args)

class Factory(object):
    """ Factory Class """

    # @ Anything declared here goes into the "dict" argument in the metaclass'  
    #   __new__ and __init__ methods. This provides a chance to pre-set the 
    #   member variables desired by the two methods, before they get run. 
    # @ This also overrides the default values declared in the meta. 
    __metaclass__ = FactoryMeta
    extra = "overridng extra"

    def selfVar(self):
        print "Instance member 'var': " + str(self.var)

    @classmethod
    def classFactory(cls, name, bases, dict):
        # @ With a factory method embedded, the Factory class can act like a 
        #   "class incubator" for generating other new classes.
        # @ The dict parameter here will later be passed to the metaclass' 
        #   __new__ and __init__, so it is the right place for setting up 
        #   member variables desired by these two methods.
        dict['class_id'] = cls.__metaclass__.count  # An ID starts from 0.
        # @ Note that this dict is for the *factory product classes*. Using 
        #   metaclass as callable is another way of writing class definition, 
        #   with the flexibility of employing dynamically generated members 
        #   in this dict.
        # @ Class' member methods can be added dynamically by using the exec 
        #   keyword on dict.
        exec(cls.extra, dict)
        exec(dict['another_func'], dict)
        return cls.__metaclass__(name + ("_%02d" % dict['class_id']), bases, dict)

    def __new__(cls, function):
        # @ Class' __new__ "creates" the instances.
        # @ This won't affect the metaclass. But it does alter the class' member
        #   as it is bounded to cls.
        cls.extra = function
        print "================================================================"
        print " Class' __new__ (\"creates\" instance objects)"
        print "----------------------------------------------------------------"
        print "Bounded to object: " + str(cls)
        print "Bounded object's __dict__: "
        pprint(DictType(cls.__dict__), depth = 1)
        print "----------------------------------------------------------------"
        print "Parameter 'function': \n" + str(function)
        print "\n"
        return super(Factory, cls).__new__(cls)

    def __init__(self, function, *args, **kwargs):
        # @ Class' __init__ initializes the instances.
        # @ Changes through self here (normally) won't affect the class or the 
        #   metaclass; they are only visible locally to the instances.
        # @ However, here you have another chance to make "static" things 
        #   visible to the instances, "locally".
        self.classFactory = self.__class__.classFactory
        print "================================================================"
        print " Class' __init__ (initiates instance objects)"
        print "----------------------------------------------------------------"
        print "Bounded to object: " + str(self)
        print "Bounded object's __dict__: "
        pprint(DictType(self.__dict__), depth = 1)
        print "----------------------------------------------------------------"
        print "Parameter 'function': \n" + str(function)
        print "\n"
        return super(Factory, self).__init__(*args, **kwargs)
# @ The metaclass' __new__ and __init__ will be run at this point, where the 
#   (manual) class definition hitting its end.
# @ Note that if you have already defined everything well in a metaclass, the
#   class definition can go dummy with simply a class name and a "pass".
# @ Moreover, if you use class factories extensively, your only use of a 
#   manually defined class would be to define the incubator class.

The output looks like this (adapted for better demonstration):

================================================================
 Metaclass's __new__ (creates class objects)
----------------------------------------------------------------
Bounded to object: <class '__main__.FactoryMeta'>
Bounded object's __dict__: 
{ ...,
 'clsVar': <function clsVar at 0x00000000029BC828>,
 'count': 1,
 'extra': 'default extra',
 'metaVar': <classmethod object at 0x00000000029B4B28>,
 'var': "Change made to <class '__main__.FactoryMeta'> by metaclass' __new__"}
----------------------------------------------------------------
Parameter 'name': Factory
Parameter 'bases': (<type 'object'>,)
Parameter 'dict': 
{ ...,
 'classFactory': <classmethod object at 0x00000000029B4DC8>,
 'extra': 'overridng extra',
 'new_elem': 'effective',
 'selfVar': <function selfVar at 0x00000000029BC6D8>}

================================================================
 Metaclass's __init__ (initiates class objects)
----------------------------------------------------------------
Bounded to object: <class '__main__.Factory'>
Bounded object's __dict__: 
{ ...,
 'classFactory': <classmethod object at 0x00000000029B4DC8>,
 'extra': 'overridng extra',
 'new_elem': 'effective',
 'selfVar': <function selfVar at 0x00000000029BC6D8>,
 'var': "Change made to <class '__main__.Factory'> by metaclass' __init__"}
----------------------------------------------------------------
Parameter 'name': Factory
Parameter 'bases': (<type 'object'>,)
Parameter 'dict': 
{ ...,
 'classFactory': <classmethod object at 0x00000000029B4DC8>,
 'extra': 'overridng extra',
 'init_elem': 'defective',
 'new_elem': 'effective',
 'selfVar': <function selfVar at 0x00000000029BC6D8>}

The calling sequence is metaclass' __new__ then its __init__. __call__ won't be called at this time.

And if we create an instance,

func1 = (
    "def printElems(self):\n"
    "   print \"Member new_elem: \" + self.new_elem\n"
    "   print \"Member init_elem: \" + self.init_elem\n"
    )
factory = Factory(func1)

The output is:

================================================================
 Metaclass's __call__ (initiates instance objects)
----------------------------------------------------------------
Bounded to object: <class '__main__.Factory'>
Bounded object's __dict__: 
{ ...,
 'classFactory': <classmethod object at 0x00000000029B4DC8>,
 'extra': 'overridng extra',
 'metaVar': <bound method type.metaVar of <class '__main__.FactoryMeta'>>,
 'new_elem': 'effective',
 'selfVar': <function selfVar at 0x00000000029BC6D8>,
 'var': "Change made to <class '__main__.Factory'> by metaclass' __call__"}

================================================================
 Class' __new__ ("creates" instance objects)
----------------------------------------------------------------
Bounded to object: <class '__main__.Factory'>
Bounded object's __dict__: 
{ ...,
 'classFactory': <classmethod object at 0x00000000029B4DC8>,
 'extra': 'def printElems(self):\n   print "Member new_elem: " + self.new_elem\n   print "Member init_elem: " + self.init_elem\n',
 'metaVar': <bound method type.metaVar of <class '__main__.FactoryMeta'>>,
 'new_elem': 'effective',
 'selfVar': <function selfVar at 0x00000000029BC6D8>,
 'var': "Change made to <class '__main__.Factory'> by metaclass' __call__"}
----------------------------------------------------------------
Parameter 'function': 
def printElems(self):
   print "Member new_elem: " + self.new_elem
   print "Member init_elem: " + self.init_elem

================================================================
 Class' __init__ (initiates instance objects)
----------------------------------------------------------------
Bounded to object: <__main__.Factory object at 0x00000000029BB7B8>
Bounded object's __dict__: 
{'classFactory': <bound method FactoryMeta.classFactory of <class '__main__.Factory'>>}
----------------------------------------------------------------
Parameter 'function': 
def printElems(self):
   print "Member new_elem: " + self.new_elem
   print "Member init_elem: " + self.init_elem

The metaclass' __call__ gets called first, then class' __new__ and __init__.

Comparing the printed members of each object, you can discover when and where they're added or changed, just as I commented in the code.

I also run the following test cases:

factory.clsVar()    # Will raise exception
Factory.clsVar()
factory.metaVar()
factory.selfVar()

func2 = (
    "@classmethod\n"
    "def printClassID(cls):\n"
    "   print \"Class ID: %02d\" % cls.class_id\n"
    )
ProductClass1 = factory.classFactory("ProductClass", (object, ), { 'another_func': func2 })

product = ProductClass1()
product.printClassID()
product.printElems()    # Will raise exception

ProductClass2 = Factory.classFactory("ProductClass", (Factory, ), { 'another_func': "pass" })
ProductClass2.printClassID()    # Will raise exception
ProductClass3 = ProductClass2.classFactory("ProductClass", (object, ), { 'another_func': func2 })

Which you can run by yourself to see how it works.

Note that I intentionally left the dynamically generated classes' names different from the variable names they assigned to. This is to display which names are actually in effect.

Another note is that I put "static" in quotes, which I refer to the concept like in C++ rather than the Python decorator. Traditionally I'm a C++ programmer, so I still like to think in its way.