12

Question: Is there a way to make a function object in python using strings?

Info: I'm working on a project which I store data in a sqlite3 server backend. nothing to crazy about that. a DAL class is very commonly done through code generation because the code is so incredibly mundane. But that gave me an idea. In python when a attribute is not found, if you define the function __getattr__ it will call that before it errors. so the way I figure it, through a parser and a logic tree I could dynamically generate the code I need on its first call, then save the function object as a local attrib. for example:

DAL.getAll()

#getAll() not found, call __getattr__

DAL.__getattr__(self,attrib)#in this case attrib = getAll

##parser logic magic takes place here and I end up with a string for a new function

##convert string to function

DAL.getAll = newFunc

return newFunc

I've tried the compile function, but exec, and eval are far from satisfactory in terms of being able to accomplish this kind of feat. I need something that will allow multiple lines of function. Is there another way to do this besides those to that doesn't involve writing the it to disk? Again I'm trying to make a function object dynamically.

P.S.: Yes, I know this has horrible security and stability problems. yes, I know this is a horribly in-efficient way of doing this. do I care? no. this is a proof of concept. "Can python do this? Can it dynamically create a function object?" is what I want to know, not some superior alternative. (though feel free to tack on superior alternatives after you've answered the question at hand)

martineau
  • 119,623
  • 25
  • 170
  • 301
Narcolapser
  • 5,895
  • 15
  • 46
  • 56
  • "exec, and eval are far from satisfactory in terms of being able to accomplish this kind of feat"? What? `exec` does that perfectly. What **specific** error or problem are you having? – S.Lott May 23 '11 at 13:47
  • 1
    try returning something from exec. – Narcolapser May 23 '11 at 13:50
  • `exec` just runs stuff, they do what they say on the tin. `a = eval("123")` will bind 123 to `a`. You get the value of the expression back. – Skurmedel May 23 '11 at 13:53
  • wait. I may not have seen eval for all it's potential then. but i still can't do things like imports and returns. integral parts of what i'm trying to do. – Narcolapser May 23 '11 at 13:55
  • In the case of `exec` you can use a callback or likewise to return values. – Skurmedel May 23 '11 at 13:56
  • @Narcolapser: Then you'll probably need something like the `codeop` library, or `py_compile`. For example: `exec(codeop.compile_command("import os; print(os.getcwd())"))` – Skurmedel May 23 '11 at 14:00
  • that's getting there. my one remaining complaint is i still don't end up with a function object. I'm going to bed now. what you have is almost exactly right. if you can tell me how to get a return statement and a function object out of this approach then put it as an answer. if you know for a fact that it can't be done, put that as an answer. truth be told, proving that the concept can't be done is an acceptable answer for me in this case. – Narcolapser May 23 '11 at 14:11
  • 2
    @Narcolapser: Exec doesn't return anything. It creates the function. After `exec` the function exists and you can use it. – S.Lott May 23 '11 at 14:32

3 Answers3

27

The following puts the symbols that you define in your string in the dictionary d:

d = {}
exec "def f(x): return x" in d

Now d['f'] is a function object. If you want to use variables from your program in the code in your string, you can send this via d:

d = {'a':7}
exec "def f(x): return x + a" in d

Now d['f'] is a function object that is dynamically bound to d['a']. When you change d['a'], you change the output of d['f']().

martineau
  • 119,623
  • 25
  • 170
  • 301
Gurgeh
  • 2,130
  • 15
  • 28
  • 2
    And as S.Lott stated, if you just run exec without the "in", you will get a new function in your current namespace. exec "def f(x): return x" will give you a new function object f, ready to be used in your code. – Gurgeh May 23 '11 at 14:42
  • that is quite what i was looking for. i guess what I was missing was i didn't think of putting 'def' into my exec statement. but this is exactly what I wanted. thanks. – Narcolapser May 23 '11 at 21:13
  • 1
    This doesn't work if x is replaced with sin(x), even after "from math import *". Seems it only works with simple math operations like x + 1.0. – Taozi Aug 01 '15 at 02:54
  • 1
    @Taozi It works fine. Your scope does not get magically transfered to d, you have to add the names that you want to use, like so: d = {'a':7, 'sin':math.sin}, which will work if you have previously imported math. – Gurgeh Aug 03 '15 at 13:45
  • 4
    Note that for Python 3+, you'd want `exec(string, d)` (it's a function, like `print` is now); see this SO question about the differences in Py2 vs 3: https://stackoverflow.com/questions/15086040/behavior-of-exec-function-in-python-2-and-python-3. – Nathan Jun 18 '19 at 14:46
  • Function can be added to a class by calling ```exec("""def foo(x):\n print(x)""", self.__dict__)``` in a method, e.g. __init__() – Thomas Oct 04 '21 at 21:35
2

can't you do something like this?

>>> def func_builder(name):
...  def f():
...   # multiline code here, using name, and using the logic you have
...   return name
...  return f
... 
>>> func_builder("ciao")()
'ciao'

basically, assemble a real function instead of assembling a string and then trying to compile that into a function.

riffraff
  • 2,429
  • 1
  • 23
  • 32
0

If it is simply proof on concept then eval and exec are fine, you can also do this with pickle strings, yaml strings and anything else you decide to write a constructor for.

Jakob Bowyer
  • 33,878
  • 8
  • 76
  • 91
  • 1
    try returning something from exec. – Narcolapser May 23 '11 at 13:52
  • This is the way to go. Use an existing parser to parse your string and then have it fill in the variables within a function object. If it's a complete function, then you'll have to get a more powerful parser. – wheaties May 23 '11 at 14:51
  • Bingo, like PIL's 'tostring' and 'fromstring' methods, liking yamls object creation as well it seems fun. – Jakob Bowyer May 23 '11 at 15:58