Eval scope in Python 2 vs. 3

Question

I came across bizarre eval behavior in Python 3 - local variables aren't picked up when eval is called in a list comprehension.

def apply_op():
    x, y, z = [0.5, 0.25, 0.75]
    op = "x,y,z"
    return [eval(o) for o in op.split(",")]
print(apply_op())

It errors in Python 3:

▶ python --version
Python 3.4.3
▶ python eval.py
Traceback (most recent call last):
  File "eval.py", line 7, in <module>
    print(apply_op())
  File "eval.py", line 5, in apply_op
    return [eval(o) % 1 for o in op.split(",")]
  File "eval.py", line 5, in <listcomp>
    return [eval(o) % 1 for o in op.split(",")]
  File "<string>", line 1, in <module>
NameError: name 'x' is not defined

And it works fine in Python 2:

▶ python --version
Python 2.7.8
▶ python eval.py
[0.5, 0.25, 0.75]

Moving it outside of the list comprehension removes the problem.

def apply_op():
    x, y, z = [0.5, 0.25, 0.75]
    return [eval("x"), eval("y"), eval("z")]

Is this intended behavior, or is it a bug?

[here](http://stackoverflow.com/questions/4198906/python-list-comprehension-rebind-names-even-after-scope-of-comprehension-is-thi) is some more info about scopes and list comprehensions in p2 and p3. — Marcin, Mar 30 '15 at 00:31
The fun part is, moving it out of function scope also stops the problem. related: http://bugs.python.org/msg81898 — wim, Mar 30 '15 at 00:32
@Marcin this is not related to the scope of the loop variable leaking outside of list comprehensions. it's about the scope of the loop variable *within* the comprehension itself. — wim, Mar 30 '15 at 00:33
@koukouviou thanks for the link - I was surprised by the 2009 time stamp! — PattimusPrime, Mar 30 '15 at 00:34
I understand the evils of `eval`, but that shouldn't be a reason to half-support a built-in. Seems like it should either be fixed or deprecated. — PattimusPrime, Mar 30 '15 at 01:00

score 25 · Accepted Answer · answered Mar 30 '15 at 00:34

25

There is a closed issue in the bug tracker for this: Issue 5242.

The resolution for this bug is won't fix.

Some comments from the Issue read:

This is expected, and won't easily fix. The reason is that list comprehensions in 3.x use a function namespace "under the hood" (in 2.x, they were implemented like a simple for loop). Because inner functions need to know what names to get from what enclosing namespace, the names referenced in eval() can't come from enclosing functions. They must either be locals or globals.

eval() is probably already an hack, there's no need to add another hack to make it work. It's better to just get rid of eval() and find a better way to do what you want to do.

answered Mar 30 '15 at 00:34

koukouviou

820
13
23

2

Yep, I can reproduce the same behavior in Python 2 using an explicit inner function. Interestingly, though, both Perl and JavaScript, which also support inner functions, *don't* exhibit this behavior: `function foo() { var x = 42; function bar() { return eval("x") }; return bar }; foo()()` yields 42 as expected, so the requirement that "inner functions need to know what names to get from what enclosing namespace" is really a quirk of Python, not a general limitation of inner functions. – Ilmari Karonen Mar 30 '15 at 12:58
1

It's really unfortunate that someone would say "this is expected". I don't expect this. One fix is for cpython to create an anonymous function that sees all of the caller's local variables, but if not then the documentation should be updated. – Neil G Apr 01 '15 at 22:12
@NeilG: By the time `eval` is called, it's too late to make decisions about closure variable availability. Supporting this would require compile-time detection of `eval`, like the hack used for 0-args `super`, with similar consequences to how if you do `supper = super`, you can't use 0-args `supper`. That, or every function with an explicit or implicit nested function would have to use the closure variable mechanism for all its variables and make all of them available to all nested functions, slowing down a big chunk of all Python variable resolution. – user2357112 Apr 05 '17 at 17:39

score 4 · Answer 2 · answered Mar 30 '15 at 06:50

If you want:

def apply_op():
    x, y, z = [0.5, 0.25, 0.75]
    op = "x,y,z"
    return [eval(o) for o in op.split(",")]
print(apply_op())

to work you'll need to capture the locals and globals as the issue is that eval(o) is the same has eval(o, globals(), locals()) but as the eval appears within the generator function the results of those functions aren't the same as they were when the eval didn't have a wrapping function so capture them outside the generator and use them inside:

def apply_op():
    x, y, z = [0.5, 0.25, 0.75]
    op = "x,y,z"
    _locals = locals()
    _globals = globals()
    return [eval(o, _globals, _locals) for o in op.split(",")]
print(apply_op())

Or better as x,y,z are locals and the strings are only variable names:

def apply_op():
    x, y, z = [0.5, 0.25, 0.75]
    op = "x,y,z"
    _locals = locals()
    return [_locals[o] for o in op.split(",")]
print(apply_op())

Eval scope in Python 2 vs. 3

2 Answers2

Linked