python string eval string got error in dict

Question

In [33]: def select_func(df):
    ...:     df=df.copy()
    ...:     rules=["df['Age']<60"]
    ...:     s={ f"n{i}" :eval(r) for i,r in enumerate(rules) }
    ...:     return df

In [34]: select_func(df=df1)
---------------------------------------------------------------------------
NameError                                 Traceback (most recent call last)
<ipython-input-34-4cc0569a2417> in <module>
----> 1 select_func(df=df1)

<ipython-input-33-0613bd096f62> in select_func(df)
      2     df=df.copy()
      3     rules=["df['Age']<60"]
----> 4     s={ f"n{i}" :eval(r) for i,r in enumerate(rules) }
      5     return df
      6

<ipython-input-33-0613bd096f62> in <dictcomp>(.0)
      2     df=df.copy()
      3     rules=["df['Age']<60"]
----> 4     s={ f"n{i}" :eval(r) for i,r in enumerate(rules) }
      5     return df
      6

<string> in <module>

NameError: name 'df' is not defined

df1=pd.DataFrame({'Name':['JOHN','ALLEN','BOB','NIKI','CHARLIE','CHANG'],
              'Age':[35,42,63,29,47,51],
              'Salary_in_1000':[100,93,78,120,64,115],
             'FT_Team':['STEELERS','SEAHAWKS','FALCONS','FALCONS','PATRIOTS','STEELERS']})

It is strange that eval() in dict got an error. Strangely enough, it works without dict.

Note：df1 is a must. I do not want to only use df as an input variable. df1 is also necessary for my application.
My Goal is that creating columns telling why the row is not selected, which refers this post. But I don't want to create n1,n2,…… to refer each condition. Instead, I want to using raw conditions as variable.
env

Python 3.7.9 | packaged by conda-forge | (default, Dec 9 2020, 20:36:16) [MSC v.1916 64 bit (AMD64)] Type 'copyright', 'credits' or 'license' for more information IPython 7.32.0 -- An enhanced Interactive Python. Type '?' for help.

@user2357112 it's an example, I delete other conditions. There are more than 15 rules. I want to use this post https://stackoverflow.com/a/67362201/5576930 , but not using n1, n2…… — Jack, Jul 15 '22 at 02:12
There's nothing in that post about using `eval`. [Read](https://nedbatchelder.com/blog/201206/eval_really_is_dangerous.html). — MattDMo, Jul 15 '22 at 02:15
@IgnatiusReilly: You likely assigned something to the `df` name at global scope. This code should produce an error. — user2357112, Jul 15 '22 at 02:25
@IgnatiusReilly, I think the problem is that there are many different names like df. — Jack, Jul 15 '22 at 02:29
Don't know why, but it seems to work if you don't use dict comprehension. — Ignatius Reilly, Jul 15 '22 at 02:42

hellohawaii · Accepted Answer · 2022-07-15T09:28:51.553

The minimal example

The minimal example of your problem is:

m1=1

def outer_fun(m):
    commands = ["m"]
    s = {i: eval(r) for i, r in enumerate(commands)}

outer_fun(m1)

Which gives:

Traceback (most recent call last):
  File "/tmp/pycharm_project_53/eval.py", line 7, in <module>
    outer_fun(m1)
  File "/tmp/pycharm_project_53/eval.py", line 5, in outer_fun
    s = {i: eval(r) for i, r in enumerate(commands)}
  File "/tmp/pycharm_project_53/eval.py", line 5, in <dictcomp>
    s = {i: eval(r) for i, r in enumerate(commands)}
  File "<string>", line 1, in <module>
NameError: name 'm' is not defined

Simplify the dict comprehension operation

The dict comprehension makes the problem complex and conceals the essence, let's get rid of it.

According to the PEP 274, the expression in you question is equivalent to s = dict([(i, eval(r)) for i, r in enumerate(commands)]), where the [(i, eval(r)) for i, r in enumerate(commands)] causes the problem.
I think the list comprehesion used above will use lambda function.You can get rid of the list comprehension, perhpas get something like s = list(map(lambda i, r: (i, eval(r)), range(len(commands)), commands)) (EDIT: this is not accurate, see my question Is list comprehension implemented via map and lambda function?, they are not the same, but they work in a similar way; Perhaps one can still uses the lambda to understand the behavior of list comprehension if doesn't want to refer to python assembly). And just applying the lambda function (lambda i, r: (i, eval(r)))(0, "m") will result in problems!
Using an explict function definition to replace the lambda function that cause the problem. which gives:

m1=1

def outer_fun(m):
    commands = ["m"]
    # All of the following lines fails:
    # s = {i: eval(r) for i, r in enumerate(commands)}
    # s = dict([(i, eval(r)) for i, r in enumerate(commands)])
    # s = [(i, eval(r)) for i, r in enumerate(commands)]
    # s = list(map(lambda i, r: (i, eval(r)), range(len(commands)), commands))
    # (lambda i, r: (i, eval(r)))(0, "m")
    def inner_func(i, r):
        return (i, eval(r))

    inner_func(0, "m")
outer_fun(m1)

And it is just like:

m1=1

def outer_fun(m):
    def inner_func():
        return (0, eval("m"))

    inner_func()
outer_fun(m1)

A most simplified version

A most simplified version will be:

m1=1

def outer_fun(m):
    def inner_func():
        eval("m")
    inner_func()
outer_fun(m1)

Which give:

Traceback (most recent call last):
  File "/tmp/pycharm_project_53/eval.py", line 7, in <module>
    outer_fun(m1)
  File "/tmp/pycharm_project_53/eval.py", line 6, in outer_fun
    inner_func()
  File "/tmp/pycharm_project_53/eval.py", line 5, in inner_func
    eval("m")
  File "<string>", line 1, in <module>
NameError: name 'm' is not defined

Why the "most simplified version" raise exception

You can refer to the doc of eval. If arguments globals and locals are not passed to the eval, the eval is executed with the globals and locals in the environment where eval() is called. However, m is not the local variable of inner_func, nor a global variable(but m1 is a global variable).

Another thing that may be insteresting is that if you replace the eval('m') with print(m), the inner_func can print without any problem, this is because m is the local variable of the outer_func that the inner_func can access.

You can check the globals and locals with the function globals() and locals():

m1=1

def outer_fun(m):
    print("local in outer:")
    print(locals())
    print("global in outer:")
    print(globals())
    def inner_func():
        print("local in inner:")
        print(locals())
        print("global in inner:")
        print(globals())
        eval("m")
    inner_func()
outer_fun(m1)

Which gives:

local in outer:
{'m': 1}
global in outer:
{'__name__': '__main__', '__doc__': None, '__package__': None, '__loader__': <_frozen_importlib_external.SourceFileLoader object at 0x7f0d85ad0eb0>, '__spec__': None, '__annotations__': {}, '__builtins__': <module 'builtins' (built-in)>, '__file__': '/tmp/pycharm_project_53/11.py', '__cached__': None, 'm1': 1, 'outer_fun': <function outer_fun at 0x7f0d85ad6280>}
local in inner:
{}
global in inner:
{'__name__': '__main__', '__doc__': None, '__package__': None, '__loader__': <_frozen_importlib_external.SourceFileLoader object at 0x7f0d85ad0eb0>, '__spec__': None, '__annotations__': {}, '__builtins__': <module 'builtins' (built-in)>, '__file__': '/tmp/pycharm_project_53/11.py', '__cached__': None, 'm1': 1, 'outer_fun': <function outer_fun at 0x7f0d85ad6280>}

How to fix

You should store the globals and locals, and then pass them to the eval.

m1=1

def outer_fun(m):
    commands = ["m"]
    globs = globals()
    locs = locals()
    s = {i: eval(r, globs, locs) for i, r in enumerate(commands)}
    print(s)
outer_fun(m1)

Futher reading

These questions may be related: