Python inner functions

Question

In python, I can write :

def func():
    x = 1
    print x
    x+=1

    def _func():
        print x
    return _func

test = func()
test()

when I run it, the output is :

1

2

As _func has access to the "x" variable defined in func. Right...

But if I do :

def func():
    x = 1
    print x

    def _func():
        x+=1
        print x
    return _func

test = func()
test()

Then I got an error message : UnboundLocalError: local variable 'x' referenced before assignment

In this case, it seems that _func cannot "see" the "x" variable

The question is: Why does print x in the first example "see" the "x" variable, whereas the mathematical operator x+=1 throws an exception?

I don't understand why...

Answer 1

Check this answer : https://stackoverflow.com/a/293097/1741450

Variables in scopes other than the local function's variables can be accessed, but can't be rebound to new parameters without further syntax. Instead, assignment will create a new local variable instead of affecting the variable in the parent scope. For example:

Answer 2

Considering that Python is an "interpreted" language, one naturally assumes that if a variable is already defined in an outer scope that same variable should still be accessible in the inner scope. And indeed in your first example when the inner _func merely prints x it works.

The thing that's non-obvious about Python though is that this is not exactly how the scope of a variable is determined. Python analyzes at compile time which variables should be considered "local" to a scope based on whether they're assigned to within that scope. In this case assignment includes augmented assignment operators. So when Python compiles your inner _func in the second example to bytecode it sees the x += 1 and determines that x must be a local variable to _func. Except of course since its first assignment is an augmented assignment there is no x variable locally to augment and you get the UnboundLocalError.

A different way to see this might be to write something like:

def _func():
    print x
    x = 2

Here again because _func contains the line x = 2 it will treat x as a local variable in the scope of that function and not as the x defined in the outer function. So the print x should also result in an UnboundLocalError.

You can examine this in deeper detail by using the dis module to display the bytecode generated for the function:

>>> dis.dis(_func)
  2           0 LOAD_FAST                0 (x)
              3 PRINT_ITEM          
              4 PRINT_NEWLINE       

  3           5 LOAD_CONST               1 (2)
              8 STORE_FAST               0 (x)
             11 LOAD_CONST               0 (None)
             14 RETURN_VALUE

The LOAD_FAST opcode is for intended for "fast" lookups of local variables that bypasses slower, more general name lookups. It uses an array of pointers where each local variable (in the current stack frame) is associated with an index in that array, rather than going through dictionary lookups and such. In the above example the sole argument to the LOAD_FAST opcode is 0--in this case the first (and only) local.

You can check on the function itself (specifically its underlying code object) that there is one local variable used in that code, and that the variable name associated with it is 'x':

>>> _func.__code__.co_nlocals
1
>>> _func.__code__.co_varnames
('x',)

That's how dis.dis is able to report 0 LOAD_FAST 0 (x). Likewise for the STORE_FAST opcode later on.

None of this is necessary to know in order to understand variable scopes in Python, but it can be helpful nonetheless to know what's going on under the hood.

As already mentioned in some other answers, Python 3 introduced the nonlocal keyword which prevents this compile-time binding of names to local variables based on assignment to that variable in the local scope.

Answer 3

You can't rebind variables that are closed over in Python 2 (you can use nonlocal in Python 3). If I have to do what you're doing in Python 2 then I do the following workaround:

class Pointer(object):
    def __init__(self, val):
        self.val = val

def func():
    px = Pointer(1)
    print px.val
    px.val += 1

    def _func():
        print px.val
    return _func

test = func()
test()

Basically I put whatever values I need to be mutated onto an object - sometimes I use a list with one element - and then re-write the code so that all that happens is method calls. The above is actually equivalent to:

class Pointer(object):
    def __init__(self, val):
        self.val = val

def func():
    px = Pointer(1)
    print px.val
    px.__setattr__('val', px.val + 1)

    def _func():
        print px.val
    return _func

test = func()
test()