Dynamically creating a class from file

Question

I've seen these "Dynamically create a class" questions which are answered saying, "use the type() function". I'm sure I'll have to at some point but right know I'm clueless. But from what I've seen you have to already know something about the class, such as a name.

What I'm trying to do is parse an idl type of file and from that create a class which will have methods and attributes. So I have NO knowledge up front of what the class name, functions, arguments or anything will be until I parse the string.

Any ideas?

Could you provide a short example of input file and class created? — Simon Bergot, May 11 '12 at 17:56

ninjagecko · Accepted Answer · 2016-06-13T20:11:58.847

http://docs.python.org/library/functions.html#type

It's a bit hard to Google for, but you can search for python type(name, bases, dict) function examples to get:

http://www.voidspace.org.uk/python/articles/metaclasses.shtml

An excerpt from the above, which gets to the heart of your question:

The following are basically equivalent:

def __init__(self, x):
    self.x = x

def printX(self):
    print self.x

Test = type('Test', (object,), {'__init__': __init__, 'printX': printX})

and:

class Test(object):
    def __init__(self, x):
        self.x = x

    def printX(self):
        print self.x

There are two ways to create functions on the fly that I can think of. The usually-bad way is to write the code and reparse it (though done correctly, this can greatly increase performance). The sane way is to implement a function which interprets your IDL. This is called a higher-order function: http://effbot.org/pyfaq/how-do-you-make-a-higher-order-function-in-python.htm

An example of what you would write, if you cannot find an interpreter for your IDL (of if it's a custom IDL) is something like the link above, such as:

def makeMethod(idlCode):
    syntax = MyIDL.parse(idlCode)

    def newMethod(*args, **kw):
        if syntax.statementType == MyIDL.IF_STATEMENT:
            if secureLookup(mySyntaxTree.IF):
               return secureLookup(args[0]) 
            else:
               return secureLookup(args[1])
        ...

    return (syntax.methodName, newMethod)

There are many more elegant ways to expand this method, if you set up a mapping between constructs of your IDL and the syntax of *args and **kw, but this gives you the most flexibility and is the most straightforward and basic way I could think of.

Then you'd do pass in:

class DynamicIdlClass(object):
    ...

for idlObject in idlCode:
    methods = dict(makeMethod(clause) for clause in idlObject.clauses})
    methods['__init__'] = makeInitMethod(idlObject.initClause)
    idlObject = type('Test', (DynamicIdlClass,), methods)

    yield idlObject  # or idlObjectsList.push(idlObject), etc.

The code here still requires that I have knowledge that the user defines `__init__` and `printX`. The voidspace.org link was pretty interesting, a bit confusing but after I muddled through it, it was pretty close. — jiveturkey, May 11 '12 at 17:49
jnbbender: It doesn't require that the user define these functions (well perhaps in the sense you are using the phrase...). There are two ways to create functions on the fly that I can think of. The bad way is to write the code and reparse it. The sane way is to implement a function which interprets your IDL. This is called a higher-order function: http://effbot.org/pyfaq/how-do-you-make-a-higher-order-function-in-python.htm — ninjagecko, May 11 '12 at 18:00
Invoking type directly is _not_ equivalent to a class statement. The former [does not resolve the metaclass bases](https://www.python.org/dev/peps/pep-0560/#dynamic-class-creation-and-types-resolve-bases). — wim, Feb 07 '22 at 13:52

lukecampbell · Answer 2 · 2012-05-11T18:04:27.987

1

On our project we create an object that exposes a concrete method that has the ability to add callable attributes. We parse a lot of yaml files for the schema and assemble these objects dynamically.

Something similar to

def add_method(name, callable):
    setattr(self,name,callable)

But this is a very naive example but you get the idea. The schema is stored inside each dynamic class under something like __schema which is a dictionary containing kwarg prototypes and docstrings.

For the actual callables make a skeleton like

def skel(self, *args, **kwargs):
    if '_schema' in kwargs:
        parse_yml, add control statements
    handle args and kwargs based on schema

some_instance.add_method('blah',skel)

Again, these are both extremely naive examples and do not cover the majority of issues and edge cases which arise when working with this type of problem. This is also one of several methods to find a solution.

Examples:

IonObject

Object Model Generator

edited May 11 '12 at 18:04

answered May 11 '12 at 17:28

lukecampbell

14,728
4
34
32

This does not create callable attributes. It is a convenience method for calling `setattr`. This does not let you, in and of itself, define methods (which have an implicit `self` first-argument), or arbitrarily set the base classes. It only lets you do what is syntactically equivalent to `myObject.fieldName = value`. The only reason this works in the IonObject code is because it is ripping already-created values from an identical class, so methods are preserved (though they may work incorrect if not implemented with __class__, maybe even then), and the base classes are already the same. – ninjagecko May 11 '12 at 17:50
One method to perform this hack, which *will* work on methods (and if you don't need to change the baseclasses), is http://code.activestate.com/recipes/81732-dynamically-added-methods-to-a-class/#c3 - it does seem to be quite fragile though if you do any reflection/inspection. – ninjagecko May 11 '12 at 17:56
I come from c++, I would think you would want to add callable to self (which is some other object), what name is I don't know. i.e - `def add_method(self, callable, value): setattr(self, callable, value)`. Can you do this? – jiveturkey May 11 '12 at 17:58
`setattr(self` is what adds the callable attribute to self. It's not quite as simple as the example, a lot of edge cases and clobbering but it's to illustrate the pattern (we) chose to employ. If you look at the more detailed examples provided it will give you a better idea of what's involved. – lukecampbell May 11 '12 at 18:00
@jnbbender: As I explained, no you can't, without hacks like the one I linked. Python methods `myObject.method(a,b,c)` actually call `method(myObject,a,b,c)`. You can do this if you do it the proper way, which is with `type(name, bases, dict)`. – ninjagecko May 11 '12 at 18:33

score 0 · Answer 3 · answered May 11 '12 at 17:53

Rather than thinking of this as 'how do I dynamically create a class', think of it as an opportunity to use the metaprogramming capabilities of Python instead.

The first place I'd look for ideas is to the standard library source for xmlrpclib -- look specifically at how it uses __getattr__ to handle otherwise non-existent method calls on the fly.

I know that I didn't really understand why they call Python a dynamic language until I had fully come to terms with those possibilities.

Dynamically creating a class from file

3 Answers3

Linked