Function arguments not accessible in dict-comprehension

Question

Why does accessing function arguments with eval in a dict comprehension fail?

ARGS1 = ('a1', 'b1')
def foo1(a1, b1):
    return {arg:eval(arg) for arg in ARGS1}
print foo1("A1", "B1") # NameError: name 'a1' is not defined

The same thing in a list comprehension is fine:

ARGS2 = ('a2', 'b2')
def foo2(a2, b2):
    return [eval(arg) for arg in ARGS2]
print foo2("A2", "B2") # OK, print: ['A2', 'B2']

It also works well without a function :

ARGS3 = ('a3', 'b3')
a3, b3 = ("A3", "B3")
print {arg:eval(arg) for arg in ARGS3} # OK, print: ['A3', 'B3']

Or if globals are defined :

ARGS4 = ('a4', 'b4')
a4, b4 = ("A4", "B4")
def foo4():
    return [eval(arg) for arg in ARGS4]
print foo4() # OK, print: ['A4', 'B4']

This really looks like a bug, but maybe I'm missing something here.

(EDITED to include without-function & with-globals examples)

Also see [Python list comprehension rebind names even after scope of comprehension. Is this right?](http://stackoverflow.com/q/4198906) — Martijn Pieters, Sep 06 '13 at 18:30
This is not a bug, but the behaviour *was* changed in Python 3 to be more consistent. — Martijn Pieters, Sep 06 '13 at 18:31
Your first example is **not** a dict comprehension, that is just a `dict` literal. `{arg: eval(arg) for arg in ARGS}` however is. — Martijn Pieters, Sep 06 '13 at 18:32
+1 to @MartijnPieters's first comment. Another way to put it is that, to the extent that there _is_ a bug here, it's the list comprehension that's buggy (and was fixed in 3.0), while the dict comprehension is working as intended. — abarnert, Sep 06 '13 at 18:42
As a side note, the fact that `eval` gets in the way of the usual name-lookup rules (which means you actually have to understand them in detail, and often work around them, instead of just trusting that they work like magic) is one of the many reasons you almost never want to use `eval`. If you really wanted to do this, do something like `loc = locals(); return {arg: loc[arg] for arg in ARGS}`, which would get you a proper closure, and be safer, and be more explicit and readable. — abarnert, Sep 06 '13 at 18:46
@abarnert: Originally it was a compromise to make list comps fast, an option that wasn't available to generators. In the end consistency was more important. — Martijn Pieters, Sep 06 '13 at 18:47
@MartijnPieters: Yes, Guido explicitly rejected calling the difference a "bug" or a "wart" in 2.x. But if you insist on seeing something here as buggy, it would be the listcomp behavior, not anything else. It's probably a lot clearer (and less arguable) to call it an "omission" instead of a "bug", as in your answer. — abarnert, Sep 06 '13 at 18:52
@abarnert: Agreed; the inconsistency is confusing users all the time, let alone the 'leaking' of loop names. — Martijn Pieters, Sep 06 '13 at 18:54

score 5 · Accepted Answer · edited May 23 '17 at 12:03

A dict comprehension is executed in a new scope, like a function.

As such, the expression locals are limited to just those named in the loop, in this case arg. The parent function locals are not considered, because closures are only bound at compile time. Names referenced by eval() cannot make use of closures.

The following also doesn't work:

>>> ARGS = ('a', 'b')
>>> def bar(a, b):
...     def foo():
...         for arg in ARGS:
...             eval(arg)
...     return foo
... 
>>> print bar("A", "B")()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "<stdin>", line 4, in foo
  File "<string>", line 1, in <module>
NameError: name 'a' is not defined

The names a and b are not available to the inner foo function unless the compiler has determined that that function actually needs access to them:

>>> def bar2(a, b):
...     def foo():
...         a, b
...         for arg in ARGS:
...             eval(arg)
...     return foo
... 
>>> print bar2("A", "B")()
None
>>> print bar2("A", "B").func_closure
(<cell at 0x1051bac20: str object at 0x104613328>, <cell at 0x1051bacc8: str object at 0x1045971e8>)
>>> print bar2("A", "B").__code__.co_freevars
('a', 'b')

Here, the line a, b can only be referring to the parent scope locals (they are not assigned to in foo() itself), so the compiler has created closures for these and the names have become available as locals within foo().

List comprehensions in Python 2 do not have their own namespace, an omission corrected in Python 3 and not extended to dict and set comprehensions. See Python list comprehension rebind names even after scope of comprehension. Is this right?

Great explanation. I didn't expect an answer this concise to cover the fact that the new scope can take closure variables, but not if they're only referenced by `eval`, but you even explained that part. — abarnert, Sep 06 '13 at 18:43
Thanks for your solution ! I approved it as it's quite detailed, but maybe you should mention that globals are also available : s/the expression locals are limited to just those named in the loop, in this case arg/the local variables are limited to just those named in the loop, in this case arg, plus the globals/ — Lucas Cimon, Sep 10 '13 at 16:15
@LucasCimon: Globals are available to all scopes in a module. `eval()` automatically picks up the current module globals if you don't pass in an explicit globals mapping. — Martijn Pieters, Sep 10 '13 at 16:15

Function arguments not accessible in dict-comprehension

1 Answers1