Adding functions from other files to a Python class

Question

I am having trouble with this setup mainly because I am not sure what I actually want in order to solve this problem.

This is the setup

- main.py
- lib
  - __init__.py 
  - index.py
  - test.py

__init__.py has this code

import os
for module in os.listdir(os.path.dirname(__file__)+"/."):
    if module == '__init__.py' or module[-3:] != '.py':
        continue
    __import__(module[:-3], locals(), globals())
del module

main.py has this code as of now

from lib.index import *
print User.__dict__

index.py has this code

class User(object):
    def test(self):
        return "hi"
    pass

test.py has this code

class User(object):
    def tes2(self):
        return "hello"

When I execute main.py it successfully prints the method test from index.py but what I am trying to do is figure out a way where I can just create a file in the lib folder where that while has only one function in the format

class User(object):
    def newFunction(self):
       return abc

and this function should automatically be available for me in main.py

I am sure that this is not a hard thing to do but I honestly don't know what I want (what to search for to solve this) which is preventing me from researching the solution.

Answer 1

You can use a metaclass to customize class creation and add functions defined elsewhere:

import types
import os
import os.path
import imp

class PluginMeta(type):
    def __new__(cls, name, bases, dct):
        modules = [imp.load_source(filename, os.path.join(dct['plugindir'], filename))
                    for filename in os.listdir(dct['plugindir']) if filename.endswith('.py')]
        for module in modules:
            for name in dir(module):
                function = getattr(module, name)
                if isinstance(function, types.FunctionType):
                    dct[function.__name__] = function
        return type.__new__(cls, name, bases, dct)


class User(metaclass=PluginMeta):
    plugindir = "path/to/the/plugindir"

    def foo(self):
        print "foo"

user = User()
print dir(user)

Then in the plugin files, just create functions not classes:

def newFunction(self, abc):
    self.abc = abc
    return self.abc

And the metaclass will find them, turn them into methods, and attach them to your class.

Answer 2

Classes are objects, and methods are nothing more than attributes on class-objects.

So if you want to add a method to an existing class, outside the original class block, all that is is the problem of adding an attribute to an object, which I would hope you know how to do:

class User(object):
    pass

def newFunction(self):
    return 'foo'

User.newFunction = newFunction

agf's metaclass answer is basically a nifty automatic way of doing this, although it works by adding extra definitions to the class block before the class is created, rather than adding extra attributes to the class object afterwards.

That should be basically all you need to develop a framework in which things defined in one module are automatically added to a class defined elsewhere. But you still need to make a number of design decisions, such as:

1. If your externally-defined functions need auxiliary definitions, how do you determine what's supposed to get added to the class and what was just a dependency?
2. If you have more than one class you're extending this way, how do you determine what goes in which class?
3. At what point(s) in your program does the auto-extension happen?
4. Do you want to say in your class "this class has extensions defined elsewhere", or say in your extensions "this is an extension to a class defined elsewhere", or neither and somewhere bind extensions to classes externally from both?
5. Do you need to be able to have multiple versions of the "same" class with different extensions active at the same time?

A metaclass such as proposed by agf can be a very good way of implementing this sort of framework, because it lets you put all the complex code in one place while still "tagging" every class that doesn't work the way classes normally work. It does fix the answers to some of the questions I posed above, though.

Answer 3

here a working code we used in a project, I'm not sure it's the best way but it worked and there is almost no additional code to add to other files

cpu.py:

from cpu_base import CPU, CPUBase

import cpu_common
import cpu_ext

cpu_base.py:

def getClass():
    return __cpu__

def setClass(CPUClass):
    global __cpu__
    __cpu__ = CPUClass
    __classes__.append(CPUClass)

def CPU(*kw):
    return __cpu__(*kw)

class CPUBase:
    def __init__(self):
        your_init_Stuff
        # optionally a method classname_constructor to mimic __init__ for each one
        for c in __classes__:
            constructor = getattr(c, c.__name__ + '_constructor', None)
            if constructor is not None:
                constructor(self)
setClass(CPUBase)

cpu_common.py:

from cpu_base import getClass, setClass
class CPUCommon(getClass()):
    def CPUCommon_constructor(self):
        pass
setClass(CPUCommon)

cpu_ext.py:

from cpu_base import getClass, setClass

class CPUExt(getClass()):
    pass
setClass(CPUExt)

to use the class import CPU from cpu.py