Dynamic class definition with methods from multiple modules

Question

I try to create classes with quite extensive methods defined in multiple modules. I need some wrappers to ensure compatibility with other classes and cannot change my modules greatly. Here is an attempt of a minimal example:

Let my (many) modules be something like:

"""moduleA.py"""
def print_something(text):
    print(f"Here is some output from module A: {text}")

and just a second for the example:

"""moduleB.py"""
def print_something(text):
    print(f"Here is some output from module B: {text}")

If I build a dynamic class definition around many of such modules, the assignment of my method does not work as hoped:

import os
import re
from importlib import import_module

# create class for every pyd-module in current working directory
cwd = os.getcwd()
for fileName in os.listdir(cwd):
    if re.search('module.+\.py', fileName):
        moduleName = re.split('\.', fileName)[0]
        module = import_module(moduleName)

        # wrapper does something needed
        def print_wrapper(self, text):
            module.print_something(text.upper())

        className = 'ClassM'+moduleName[1:]
        dynamicClass = type(className, (object, ), {
            "print_something": print_wrapper})
        # register dynamic class for use
        globals()[dynamicClass.__name__] = dynamicClass

instanceA = ClassModuleA()
instanceA.print_something("testA")
instanceB = ClassModuleB()
instanceB.print_something("testB")

output (only from module B and not from module A):

Here is some output from module B: TESTA
Here is some output from module B: TESTB

I tried to use copy operations, but they do not work for builtin_function_or_method(s). (see: How to get builtin_function_or_method objects copied in ram without wrapping? and How can I make a deepcopy of a function in Python?)

Is there a way around this problem?

DavidW · Accepted Answer · 2021-05-24T20:28:07.017

Nothing to do with Cython. In these kind of cases it's worth testing with plain Python as well (i.e. "minimizing" your minimum reproducible example).

def print_wrapper(self, text):  # wrapper does something needed
    module.print_something(text.upper())

This says "look up the current value of module in the global scope and call its attribute print_something". The current value of module is evaluated at the point the function is called and thus is always the last one in the loop.

There's a number of techniques to capture variables at definition time, but one option might be

def print_wrapper(self, text, local_module=module):  # wrapper does something needed
    local_module.print_something(text.upper())

In the line if re.search('.pyd', fileName):, you're aware that . has a special meaning in regexes, right?

Thanks a lot! Makes sense and works fine. I was unnecessarily at a loss there. I reduced my example by cython to avoid confusion of others and corrected the unrelated regex mistake. — umfundi, May 25 '21 at 06:16

Dynamic class definition with methods from multiple modules

1 Answers1