31

EDIT: Looks like this is a very old "bug" or, actually, feature. See, e.g., this mail

I am trying to understand the Python scoping rules. More precisely, I thought that I understand them but then I found this code here:

x = "xtop"
y = "ytop"
def func():
    x = "xlocal"
    y = "ylocal"
    class C:
        print(x)
        print(y)
        y = 1
func()

In Python 3.4 the output is:

xlocal
ytop

If I replace the inner class by a function then it reasonably gives UnboundLocalError. Could you explain me why it behaves this strange way with classes and what is the reason for such choice of scoping rules?

ivanl
  • 741
  • 7
  • 17
  • 1
    It wont give any error unless you call the inner function – Padraic Cunningham Mar 29 '15 at 18:56
  • 3
    Interesting; `y` should be an unbound exception in the case of `y`, but it isn't even when you make `C` a global. Corner cases FTW! Class bodies are special; they do not create a new scope of their own. As such `y = 1` would make it a local, but until that point it is a global *in class statements **only***. – Martijn Pieters Mar 29 '15 at 18:57
  • 2
    [9.2. Python Scopes and Namespaces](https://docs.python.org/3/tutorial/classes.html#python-scopes-and-namespaces) and [4.1. Naming and binding](https://docs.python.org/3/reference/executionmodel.html#naming-and-binding) are a start, you might need to re-read them often till it sinks in. – wwii Mar 29 '15 at 18:57
  • 2
    if it's actual python3 code, then it should be `print(x)` and `print(y)`. `print x` is python2 code. – zmo Mar 29 '15 at 18:59
  • 1
    thanks, @zmo . changed that. Actually it behaves same way in Python2 – ivanl Mar 29 '15 at 19:04
  • 4
    This question could be helpful http://stackoverflow.com/questions/4296677/creating-a-class-within-a-function-and-access-a-function-defined-in-the-containi – Mazdak Mar 29 '15 at 19:19
  • Thanks @Kasra . It is strange however that the example 3 in that question works only in python3.4 and does not work in python3.1 – ivanl Mar 29 '15 at 19:28
  • 1
    ```dis.dis(func)``` with and without the ```y=1``` assignment in ```C``` shows that the former only loads a closure for ```x``` and the latter loads a closure for ```x``` and ```y``` when ```C``` gets *built*. Thought that was interesting though it doesn't point to any *rule*. – wwii Mar 29 '15 at 19:29
  • Sounds like this is a bug and should be reported as such. – Dunes Mar 29 '15 at 21:09
  • 1
    @Kasra: that is the Python **2.1** *What's New* document. That is ancient history, not a recent change. – Martijn Pieters Mar 29 '15 at 21:20
  • See also: https://stackoverflow.com/questions/9505979/the-scope-of-names-defined-in-class-block-doesnt-extend-to-the-methods-blocks – Karl Knechtel Sep 10 '22 at 09:47

2 Answers2

13

TL;DR: This behaviour has existed since Python 2.1 PEP 227: Nested Scopes, and was known back then. If a name is assigned to within a class body (like y), then it is assumed to be a local/global variable; if it is not assigned to (x), then it also can potentially point to a closure cell. The lexical variables do not show up as local/global names to the class body.


On Python 3.4, dis.dis(func) shows the following:

>>> dis.dis(func)
  4           0 LOAD_CONST               1 ('xlocal')
              3 STORE_DEREF              0 (x)

  5           6 LOAD_CONST               2 ('ylocal')
              9 STORE_FAST               0 (y)

  6          12 LOAD_BUILD_CLASS
             13 LOAD_CLOSURE             0 (x)
             16 BUILD_TUPLE              1
             19 LOAD_CONST               3 (<code object C at 0x7f083c9bbf60, file "test.py", line 6>)
             22 LOAD_CONST               4 ('C')
             25 MAKE_CLOSURE             0
             28 LOAD_CONST               4 ('C')
             31 CALL_FUNCTION            2 (2 positional, 0 keyword pair)
             34 STORE_FAST               1 (C)
             37 LOAD_CONST               0 (None)
             40 RETURN_VALUE

The LOAD_BUILD_CLASS loads the builtins.__build_class__ on the stack; this is called with arguments __build_class__(func, name); where func is the class body, and name is 'C'. The class body is the constant #3 for the function func:

>>> dis.dis(func.__code__.co_consts[3])
  6           0 LOAD_NAME                0 (__name__)
              3 STORE_NAME               1 (__module__)
              6 LOAD_CONST               0 ('func.<locals>.C')
              9 STORE_NAME               2 (__qualname__)

  7          12 LOAD_NAME                3 (print)
             15 LOAD_CLASSDEREF          0 (x)
             18 CALL_FUNCTION            1 (1 positional, 0 keyword pair)
             21 POP_TOP

  8          22 LOAD_NAME                3 (print)
             25 LOAD_NAME                4 (y)
             28 CALL_FUNCTION            1 (1 positional, 0 keyword pair)
             31 POP_TOP

  9          32 LOAD_CONST               1 (1)
             35 STORE_NAME               4 (y)
             38 LOAD_CONST               2 (None)
             41 RETURN_VALUE

Within the class body, x is accessed with LOAD_CLASSDEREF (15) while y is load with LOAD_NAME (25). The LOAD_CLASSDEREF is a Python 3.4+ opcode for loading values from closure cells specifically within class bodies (in previous versions, the generic LOAD_DEREF was used); the LOAD_NAME is for loading values from locals and then globals. However closure cells show up neither as locals nor globals.

Now, because the name y is stored to within the class body (35), it is consistently being used as not a closure cell but a local/global name. The closure cells do not show up as local variables to the class body.

This behaviour has been true ever since implementing PEP 227 - nested scopes. And back then BDFL stated that this should not be fixed - and thus it has been for these 13+ years.


The only change since PEP 227 is the addition of nonlocal in Python 3; if one uses it within the class body, the class body can set the values of the cells within the containing scope:

x = "xtop"
y = "ytop"
def func():
    x = "xlocal"
    y = "ylocal"
    class C:
        nonlocal y  # y here now refers to the outer variable
        print(x)
        print(y)
        y = 1

    print(y)
    print(C.y)

func()

The output now is

xlocal
ylocal
1
Traceback (most recent call last):
  File "test.py", line 15, in <module>
    func()
  File "test.py", line 13, in func
    print(C.y)
AttributeError: type object 'C' has no attribute 'y'

That is, print(y) read the value of the cell y of the containing scope, and y = 1 set the value in that cell; in this case, no attribute was created for the class C.

7

First focus on the case of a closure -- a function within a function:

x = "xtop"
y = "ytop"
def func():
    x = "xlocal"
    y = "ylocal"
    def inner():
 #       global y
        print(x)
        print(y)
        y='inner y'
        print(y)
    inner()  

Note the commented out global in inner If you run this, it replicates the UnboundLocalError you got. Why?

Run dis.dis on it:

>>> import dis
>>> dis.dis(func)
  6           0 LOAD_CONST               1 ('xlocal')
              3 STORE_DEREF              0 (x)

  7           6 LOAD_CONST               2 ('ylocal')
              9 STORE_FAST               0 (y)

  8          12 LOAD_CLOSURE             0 (x)
             15 BUILD_TUPLE              1
             18 LOAD_CONST               3 (<code object inner at 0x101500270, file "Untitled 3.py", line 8>)
             21 LOAD_CONST               4 ('func.<locals>.inner')
             24 MAKE_CLOSURE             0
             27 STORE_FAST               1 (inner)

 14          30 LOAD_FAST                1 (inner)
             33 CALL_FUNCTION            0 (0 positional, 0 keyword pair)
             36 POP_TOP
             37 LOAD_CONST               0 (None)
             40 RETURN_VALUE

Note the different access mode of x vs y inside of func. The use of y='inner y' inside of inner has created the UnboundLocalError

Now uncomment global y inside of inner. Now you have unambiguously create y to be the top global version until resigned as y='inner y'

With global uncommented, prints:

xlocal
ytop
inner y

You can get a more sensible result with:

x = "xtop"
y = "ytop"
def func():
    global y, x
    print(x,y)
    x = "xlocal"
    y = "ylocal"
    def inner():
        global y
        print(x,y)
        y = 'inner y'
        print(x,y)
    inner()    

Prints:

xtop ytop
xlocal ylocal
xlocal inner y

The analysis of the closure class is complicated by instance vs class variables and what / when a naked class (with no instance) is being executed.

The bottom line is the same: If you reference a name outside the local namespace and then assign to the same name locally you get a surprising result.

The 'fix' is the same: use the global keyword:

x = "xtop"
y = "ytop"
def func():
    global x, y
    x = "xlocal"
    y = "ylocal"
    class Inner:
        print(x, y)
        y = 'Inner_y'
        print(x, y) 

Prints:

xlocal ylocal
xlocal Inner_y

You can read more about Python 3 scope rules in PEP 3104

wim
  • 338,267
  • 99
  • 616
  • 750
dawg
  • 98,345
  • 23
  • 131
  • 206
  • 5
    I understand the behavior for functions you pointed out. But it is still not clear what is the reason for _classes_ to behave differenly. – ivanl Mar 29 '15 at 19:41
  • Classes will be similar but you need to further qualify instance vs class variables and nested class and nested instance variables. Bottom line is the same: The presence of the assignment of `y=1` make the reference `print(y)` above the assignment ambiguous inside the closure. – dawg Mar 29 '15 at 20:37
  • I would understand if my code would fail with `UnboundLocal` , but it looks really confusing how `y` can "bypass" y_local. Not failing with `UnboundLocal` really looks like a bug – ivanl Mar 29 '15 at 20:51
  • 1
    There is a deference in how the interpreter treats each case, true. It is unusual to just have a class definition and execute code by its presence though. – dawg Mar 29 '15 at 20:58
  • 4
    PEP 3104 only introduces the `nonlocal` statement and doesn't actually explain the behaviour. The behaviour is a long-standing PEP 227 side-effect, 3104 never altered that. – Martijn Pieters Mar 29 '15 at 21:24
  • @MartijnPieters: I did not claim 3104 was definitive -- only the most recent. It references PEP 227 and has a general overview. – dawg Apr 01 '15 at 05:30