Precise rules of variable binding in nested scopes

Question

I seem to have misunderstood something about Python variable binding. What are the precise rules for deciding which variable is accessed given a nested scope with shadowing names?

Let me illustrate with some examples. First the basic shadow.

a = 1

def foo():
    a = 2
    def _foo():
        return a
    return _foo()

print(foo())  # -> 2

Everything is fine here. The value is overwritten and returned accordingly. However, if the value is changed after the function definition, it is still the inner value:

def bar():
    def _bar():
        return a
    a = 2
    return _bar()

print(bar())  # -> 2

What's more, defining a function that references a non-existent variable is possible.

def baz():
    def _baz():
        return b
    return _baz()

Then, if b is defined later, the function can be executed. But not if is defined in another inner scope:

def qux(f):
    b = 3
    return f()

print(qux(baz()))  # -> NameError

Now all of these cases could be explained by having Python know about lines that come later in the program, but that conflicts with my knowledge of Python being an interpreted language, advancing line by line. So are statements parsed at once instead of line by line?

A weird behaviour with shadowing class attributes throws me off a bit more.

class C:
    a = 2
    b = a

    def meth(self):
        return a

    c = meth

print(C.b, C().meth(), C.c)  # -> 2 1 C.meth

Here a is defined as a class attribute and is successfully used in b, but this does not carry over to the method definition. The method itself can be used in later attributes, but not for example in other methods without going through self.

Is my guess about the binding happening all at once correct? And in that case are class bodies an exception by design, or are they not a scope at all? Or is something else going on here?

Answer 1

I think you might be overthinking it.

By default, variables when created are put in the narrowest enclosing function's scope.

Variables from all enclosing scopes are available in a read-only capacity, be that an enclosing function's scope or the global scope. If you try to assign to this, it'll create a new variable in the narrowest enclosing scope, shadowing those outside. Using the global keyword to bring an external variable into the local scope will stop this from happening, allowing you to assign things to the non-local scope.

Additionally, keep in mind that functions are compiled and evaluated at the time when the def statement is interpreted. For nested functions, essentially, every new call re-evaluates the inner functions. This also means that inner functions have read-only access to the scope of the outer functions. Same rules as usual.

Your bar() example works because, by the time python tries to access the variable a, it is present in at least one of the enclosing scopes. Python doesn't check these things until the last possible moment. Your qux() example doesn't work because the scope in which b is declared does not enclose the scope where _baz() is defined, and thus is not accessible.

---

Class scopes are weird. When the class is evaluated, all variables defined inside it are bound to the class. However, the class doesn't really count as a scope of its own, for the purpose of the methods enclosed inside it. Think of meth() as an unbound function, declared in the global scope, which C.meth refers to (and, now, C.c). Calling a function via dot notation is a syntactic shorthand:

# the following two are identical
C().meth()
C.meth(C())

and while C.meth is technically bound to C, it's not enclosed in C's class-level namespace. Trying to do C().meth() will fail, because a is not defined with respect to the function. (note that if a is defined in the global scope, the function will work as expected - C.meth() has the global scope as a parent, not C's class-level scope).