10

I am updating a project from python 2.7 to python 3.6.

I have a list comprehension that looks up variables from locals which worked in python 2.7. It only works in python 3.6 when I switch to using globals.

Below is a toy example to illustrate the issue.

The relevant code is:

(A,B,C) = (1,2,3)
myvars = ['A','B','C']

If I execute the following code:

[locals().get(var) for var in myvars]

the return value in python 3.6 is:

[None, None, None]

However, the return value in python 2.7 is:

[1, 2, 3]

If I execute the following code using globals:

[globals().get(var) for var in myvars]

then I get the same result in python 2.7 and 3.6:

[1, 2, 3]

Can anyone explain why the code using locals() no longer works in python 3.6?

cs95
  • 379,657
  • 97
  • 704
  • 746
Daniel C
  • 113
  • 9

2 Answers2

8

If we dissemble a more basic list-comprehension. we can see why this is happening:

>>> import dis
>>> dis.dis("[i for i in range(10)]")
  1           0 LOAD_CONST               0 (<code object <listcomp>)
              3 LOAD_CONST               1 ('<listcomp>')
              6 MAKE_FUNCTION            0
              9 LOAD_NAME                0 (range)
             12 LOAD_CONST               2 (10)
             15 CALL_FUNCTION            1 (1 positional, 0 keyword pair)
             18 GET_ITER
             19 CALL_FUNCTION            1 (1 positional, 0 keyword pair)
             22 RETURN_VALUE

So, on line 9, we see that a list-comp actually makes a function which is called (you may be more used to this when using generators). Therefore, the function would have its own set of local variables as it is in a separate scope - thus yielding this unexpected behaviour.

Furthermore, as pointed out by @juanpa, the "What's New in Python3" docs do explicitly state the following:

list comprehensions have different semantics: they are closer to syntactic sugar for a generator expression inside a list() constructor, and in particular the loop control variables are no longer leaked into the surrounding scope.

Note also that there is a bug page on this: https://bugs.python.org/issue21161, however juanpa has convinced me that this isn't a bug for the reason that it is clearly documented to work this way!

Joe Iddon
  • 20,101
  • 7
  • 33
  • 54
5

List comprehensions define a new scope in Python 3, which they did not do in Python 2. Simply put, A, B, and C aren't local variables in the list comprehension in Python 2.

As an aside, you probably should have been using an explicit dict {'A': 1, 'B': 2, 'C': 3} instead of separate variables all along. Now is a good time to fix that.

chepner
  • 497,756
  • 71
  • 530
  • 681