What's the difference between "bind to variables" and "bind to object" in Python

Question

When I studied "naming and binding" in Python, I saw the following example:

>>> def testClosure(maxIndex):
        def closureTest(maxIndex=maxIndex):
            return maxIndex
        maxIndex += 5
        return closureTest()
>>> print(testClosure(10))
10


>>> def testClosure(maxIndex):
        def closureTest():
            return maxIndex
       maxIndex += 5
       return closureTest()
>>> print(testClosure(10))
15

The author explained it as: In the latter function, free variable in an inner scope bind to variable in the outer scope, not objects.

Then my question is: What's the difference between "bind to variable" and "bind to object" in Python?

Also, it's very tricky:the result is different if I re-arrange the code.

>>> def testClosure(maxIndex):
        maxIndex += 5
        def closureTest(maxIndex=maxIndex):
            return maxIndex
        return closureTest()

>>> print(testClosure(10))
15

Thanks in advance.

Answer 1

Two key facts:

1. Python uses the LEGB rule to look up the value of (bare) variables names. LEGB stands for Local, Extended, Global, Builtins. That means a variable name "binds" to the local value, and if there is none, then the value is looked-up in the extended scope, and if there is no such variable, lookup is done in the global scope, and finally in the builtins scope.

2. When defining a function like

   def closureTest(maxIndex=maxIndex):
       return maxIndex
   

   default values are fixed at definition-time, not run-time. By definition-time I mean the time when the def statement is processed -- when the function is defined. By run-time I mean the time when the function is called. Note that when you have nested functions, the inner function's definition-time occurs after the outer function has been called.

---

The first example is made more complicated by the fact that the variable name maxIndex is overused. You'll understand the first example if you first understand this:

>>> def testClosure(maxIndex):              
        def closureTest(index=maxIndex):     # (1)
            return index                     
        maxIndex += 5
        return closureTest()                 # (2)
>>> print(testClosure(10))

• (1) At definition-time, index's default value is set to 10.
• (2) When closureTest() is called with no arguments, index is set to the default value 10. So this is the value returned.

---

def testClosure(maxIndex):
    def closureTest():
        return maxIndex                 # (3)
   maxIndex += 5
   return closureTest()                 # (4)
print(testClosure(10))

• (3) The LEGB rule tells Python to lookup the value of maxIndex in the local scope. There is no maxIndex defined in the local scope, so it looks in the extended scope. It finds the maxIndex which is an argument to testClosure.

• (4) By the time closureTest() is called, maxIndex has the value 15. So the maxIndex returned by closureTest() is 15.

---

>>> def testClosure(maxIndex):
        maxIndex += 5                           # (5)    
        def closureTest(maxIndex=maxIndex):     # (6)
            return maxIndex
        return closureTest()                    # (7)

• (5) maxIndex is 15

• (6) closureTest's maxIndex is set to have default value 15 at definition-time.

• (7) When closureTest() is called with no arguments, the default value for maxIndex is used. The value 15 is returned.

Answer 2

It might be less confusing if you think of the binding that's happening in the parameter expression of the 'def' statement. When you see 'def closureTest(maxIndex=maxIndex):' that is a statement like 'if' or 'while' which is followed by a suite of code to be parsed and bound to the function (callable object).

The 'def' statement is evaluated in the scope where it's found (conceptually at the same level of nesting/indentation). Its parameter expression declares how arguments will be mapped to names within the function's own scope. Any of those which provide a default (such as maxIndex in your examples) create a function object with the corresponding parameter name bound to whatever object was named or instantiated at the time (within the scope of) the 'def' statement.

When the function is called each of its parameters (names within its scope)is bound to any arguments supplied to the function. Any optional parameters are thus left bound to whichever arguments were evaluated as part of the 'def' statement.

In all of your examples an inner function is created during each invocation of the outer function. In your second example the parameter list is empty and the inner function is simply seeing the name through one level of nested scope. In the first example the inner function's def statement creates a default maxIndex name within the new function's namespace (thus preventing any resolution of the name with values from the surrounding scope, as you'd expect for any local variable within any function).

In the last example the value of maxIndex is modified before the inner function is (re)-defined. When you realize the the function is being (re)-defined on each outer function invocation then it shouldn't seem so tricky.

Python is a dynamic language. 'def' is a statement is being executed by the VM every time the flow of control passes through that line of your code. (Yes, the code has been byte compiled, but 'def' is compiled into VM op codes which perform code evaluation and name binding (to the function's name) at run-time).

If you define a function with a parameter list like '(myList=list())' then a list will be instantiated as the definition is executed. It will be accessible from within invocations of the functions code any time the function is called with no arguments. Any invocation with an argument will be executed with that parameter name bound to the argument supplied at invocation. (The object instantiated at def time is still referenced by the code object that was defined -- the suite indented after the def statement).

None of this will make any sense if you don't keep the distinction between parameters and arguments. Remember that parameters are part of the function's definition; they define how arguments will be mapped into the function's local namespace. Arguments are part of the invocation; they are the things being passed into any call of the function.

I hope this helps. I realize that the distinction is subtle and the terms are very frequently mis-used as though they were interchangeable (including throughout the Python documentation).

Answer 3

>>> def testClosure(maxIndex):
        def closureTest(maxIndex=maxIndex): # You're creating a function with a kwarg of maxIndex
            return maxIndex                 # which references the int passed in from outside
        maxIndex += 5                       # Incrementing maxIndex means that the maxIndex in the outer
        return closureTest()                # scope now points to a different int.
>>> print(testClosure(10))
10

Consider:

>>> a = b = 1
>>> a += 1
>>> a
2
>>> b
1

I don't know what you mean by "bind to the object" and "bind to variable". "Variables" are references to things, when you increment a, you're modifying it to refer to a different value, b still refers the to original value.

When you're not passing in the value of maxIndex into your inner function and then ask for it, as it's not defined in the local scope, it's looked for in the extended scope.

I'm making a realively large assumption here, but you can see a difference in the time it takes to execute, which I'd be inclined to attribute to the cost of this extended lookup:

>>> import timeit
>>> def testClosure(maxIndex):
...     def closureTest(maxIndex=maxIndex):
...         return maxIndex
...     maxIndex += 5
...     return closureTest()
...
>>> def testClosure2(maxIndex):
...     def closureTest():
...         return maxIndex
...     maxIndex += 5
...     return closureTest()
...
>>> timeit.Timer('testClosure(10)','from __main__ import testClosure').timeit()
1.4626929759979248
>>> timeit.Timer('testClosure2(10)','from __main__ import testClosure2').timeit()
1.7869210243225098