Calling user-defined function as native | Python

Question

I am building a system which is much like Mathwork's Simulink or Tanner-Spice where the user defines a function using available operators. Then I need to run that function and return the response to the user. I am using Javascript for UI interaction. Internally, the user-defined function (UDF) is captured as a JSON and passed to a Python server that parses this JSON.

My problem is, how to I now run this UDF? I am not worried about malicious users exploiting this ability to hack, because all my users are trusted ones.

One way I thought was to write the UDF onto disk as a python script and then running the commands.getstatusoutput(). The problem here is that the function might expect several inputs, and it is not possible to pass those.

What I am looking for is the ability to dynamically loading a new python file and its functions, and be able to call those.

Found a blog post that explains how to do this. I guess the problem was me not using the right keywords to search.

Anyway, David Janes' blog here explains how to dynamically load a python script.

I would still invite you guys to comment and propose if there are better ways of doing what I wanted.

Thanks, Nik

Is the UDF a Python callable? If so, you can marshal/pickle it as is done in [this answer](http://stackoverflow.com/questions/6234586/we-need-to-pickle-any-sort-of-callable). — ezod, Aug 26 '12 at 19:28
Thanks for your comment @ezod. I have never used marshal/pickle, and the basic documentation looks a little involved. It might be useful in my case, though I'll have to go into deeper details to verify it. — Nik, Aug 26 '12 at 19:50
Even if you trust your users, that doesn't mean you can trust input. If you do, you'll leave them and your site open to [CSRF](https://www.owasp.org/index.php/CSRF) exploits and the like. — outis, Dec 22 '12 at 22:52

score 1 · Answer 1 · answered Aug 27 '12 at 01:14

Here's a simple class that creates module-like objects out of code strings, files or code objects:

class DynamicModule(object):
    def __init__(self, code):
        exec code in self.__dict__

Example usage:

>>> mod = DynamicModule("""
... def foo(x, y):
...     print x**2 + y
... """)
>>> 
>>> mod.foo(10, 20)

Example with a file (assuming /tmp/hello.py contains a function called hello):

>>> mod2 = DynamicModule(open('/tmp/hello.py'))
>>> mod2.hello('World')
Hello, World!

IT Ninja · Answer 2 · 2012-08-26T20:27:25.757

0

You could use the exec module for this, since the input is trusted.

exec documentation: http://docs.python.org/reference/simple_stmts.html#exec

Documentation excerpt:

This statement supports dynamic execution of Python code. The first expression should evaluate to either a string, an open file object, or a code object. If it is a string, the string is parsed as a suite of Python statements which is then executed (unless a syntax error occurs). [1] If it is an open file, the file is parsed until EOF and executed. If it is a code object, it is simply executed. In all cases, the code that’s executed is expected to be valid as file input (see section File input).

However, you should note that when using exec you may not use return or yield statements outside of functions.

Example:

your_json_data="def example(arg1,arg2):    print arg1,arg2"
exec(your_json_data)
example("Hello","World")
##Output: "Hello World"

edited Aug 26 '12 at 20:27

answered Aug 26 '12 at 19:48

IT Ninja

6,174
10
42
65

Thanks for pointing out `exec`. Though in my case, this would probably not work. I am running a server (say myserver). It has a certain data structure (say an array) which it wants to pass to a UDF (say `userfunc()`) which is saved in a python file (say `udfile.py`). I want to be able to call `userfunc(array)`. `exec` lets me to only _run_ the `udfile.py` but not _load_ it as a module and call `userfunc` with arguments. Please correct me if I am wrong. – Nik Aug 26 '12 at 20:12
do you have to save it as a file? you could just `exec` the data straight from the `json` data and you can use it just as you would a module. I will add an example to my answer. – IT Ninja Aug 26 '12 at 20:25

score 0 · Answer 3 · edited May 23 '17 at 12:31

To import a Python file with a fixed name that in a sys.path:

import mod # it can import mod.py file

result = mod.some_function(*args)

To import module if its name is in a string:

import importlib

m = importlib.import_module("mod")
result = m.some_function(*args)

if you have the module's content in a string:

ns = {}
exec """

def some_function(a, b):
    return a + b

# other functions, classes or any code
""" in ns

result = ns['some_function'](1, 2)
print result # -> 3

If you don't control the input completely then you should execute the above code in a restricted environment e.g., you could send the strings to a sandboxed pypy interpreter.

Python can parse Python

There is ast module which can help you parse and manipulate code in a safe manner, example: Evaluating a mathematical expression in a string.

Calling user-defined function as native | Python

3 Answers3

Python can parse Python