setting imported functions as members in a static dictionary

Question

There is a simple class where I want to store some functions statically in a dictionary using different ways:

import os, sys
class ClassTest():
    testFunc = {}
    def registerClassFunc(self,funcName):
        ClassTest.testFunc[funcName] = eval(funcName)
    @classmethod
    def registerClassFuncOnClass(cls,funcName):
        cls.testFunc[funcName] = eval(funcName)
    @staticmethod
    def registerClassFuncFromStatic(funcName):
        ClassTest.testFunc[funcName] = eval(funcName)

Some example methods:

def user_func():
    print("I run therefore I am self-consistent")
def user_func2():
    print("I am read therefore I am interpreted")
def user_func3():
    print("I am registered through a meta function therefore I am not recognized")
def user_func4():
    print("I am registered through an instance function therefore I am not recognized")
def user_func5():
    print("I am registered through a static function therefore I am not recognized")

And a little test:

if __name__ == "__main__":
    a = ClassTest()
    a.testFunc["user_func"] = user_func
    a.testFunc["user_func"]()
    a.testFunc["user_func2"] = eval("user_func2")
    a.testFunc["user_func2"]()

    ClassTest.testFunc["user_func"] = user_func
    ClassTest.testFunc["user_func"]()
    ClassTest.testFunc["user_func2"] = eval("user_func2")
    ClassTest.testFunc["user_func2"]()

    a.registerClassFunc("user_func5")  # does not work on import
    a.testFunc["user_func5"]()
    ClassTest.registerClassFuncFromStatic("user_func3") # does not work on import
    ClassTest.testFunc["user_func3"]()
    ClassTest.registerClassFuncOnClass("user_func4") # does not work on import
    ClassTest.testFunc["user_func4"]()

All this works provided all these elements are in the same file. As soon as the functionality is split up in 2 files and a main file:

from ClassTest import ClassTest
from UserFunctions import user_func,user_func2, user_func3, user_func4, user_func5
if __name__ == "__main__":
    a = ClassTest()
    a.testFunc["user_func"] = user_func
    ...

Only the first two keep working (setting the function directly), the others - using a function to do the same thing - give a NameError on all the eval calls. For instance: NameError: name 'user_func5' is not defined.

What is the logic here for the loss of scope when using the methods versus directly setting the functions? And can I get it to work using imports from other packages so I can place any function in the class with a method rather than directly?

score 1 · Accepted Answer · edited Jun 20 '20 at 09:12

There's a live version of fix #1 from this answer online that you can try out for yourself

The problem

You're right that the reason this doesn't work is due to scoping issues. You can figure out what's going on by scrutinizing the docs for eval:

eval(expression, globals=None, locals=None)

...If both dictionaries [ie globals and locals] are omitted, the expression is executed in the environment where eval() is called.

Thus, it is reasonable to assume that the issue you're having is down to the contents of globals and locals in the context (ie within the definition (and possibly separate module) of ClassTest) in which eval is getting called. Since the context in which eval is getting called is not, in general, the context in which you have defined and/or imported user_func, user_func2...., these functions are undefined as far as eval is concerned. This line of thinking is backed up by the docs for globals:

globals()

...This is always the dictionary of the current module (inside a function or method, this is the module where it is defined, not the module from which it is called).

The fix

You have a few different options for how you can go about fixing this code. All of them are going to involve passing locals from the context in which you call, eg, ClassTest.registerClassFunc to the context in which that method is defined. Additionally, you should take the opportunity to factor out the use of eval from your code (its use is considered bad practice, it's a massive security hole, yadda yadda yadda). Given that locals is the dict of a scope in which user_func is defined, you can always just do:

locals['user_func']

instead of:

eval('user_func')

fix #1

Link to live version of this fix

This'll be the easiest fix to implement, since it only requires a few tweaks to the definitions of the methods of ClassTest (and no changes to any method signatures). It relies on the fact that it is possible to use the inspect package within a function to directly grab the locals of the calling context:

import inspect

def dictsGet(s, *ds):
    for d in ds:
        if s in d:
            return d[s]
    # if s is not found in any of the dicts d, treat it as an undefined symbol
    raise NameError("name %s is not defined" % s)

class ClassTest():
    testFunc = {}
    def registerClassFunc(self, funcName):
        _frame = inspect.currentframe()
        try:
            _locals = _frame.f_back.f_locals
        finally:
            del _frame

        ClassTest.testFunc[funcName] = dictsGet(funcName, _locals, locals(), globals())

    @classmethod
    def registerClassFuncOnClass(cls, funcName):
        _frame = inspect.currentframe()
        try:
            _locals = _frame.f_back.f_locals
        finally:
            del _frame

        cls.testFunc[funcName] = dictsGet(funcName, _locals, locals(), globals())

    @staticmethod
    def registerClassFuncFromStatic(funcName):
        _frame = inspect.currentframe()
        try:
            _locals = _frame.f_back.f_locals
        finally:
            del _frame

        ClassTest.testFunc[funcName] = dictsGet(funcName, _locals, locals(), globals())

If you use the above given definition of ClassTest, the import test you cooked up will now function as expected.

Pros

Provides exactly the originally intended functionality.
Involves no changes to function signatures.

Cons

Calling inspect.currentframe() can cause a dip in performance, so you may not be able to use this fix if you're planning on calling the methods of ClassTest a million times a second.
inspect.currentframe() is only guaranteed to work on CPython. Mileage may vary when running this code with other implementations of Python.

fix #2

Fix #2 is basically the same as fix #1, except that in this version you explicitly pass locals into the methods of ClassTest at the point of call. For example, under this fix the definition of ClassTest.registerClassFunc would be:

def registerClassFunc(self, funcName, _locals):
        ClassTest.testFunc[funcName] = dictsGet(funcName, _locals, locals(), globals())

and you would call it in your code like this:

a = ClassTest()
a.registerClassFunc("user_func5", locals())

Pros

Doesn't rely on inspect.currentframe(), and so is probably more performant/portable than fix #1.

Cons

You have to modify method signatures, so you'll also have to change any existing code that uses those methods.
You'll have to add the locals() boilerplate to every invocation of every ClassTest method from here on.

setting imported functions as members in a static dictionary

1 Answers1

The problem

The fix

fix #1

Pros

Cons

fix #2

Pros

Cons