Python lambda list: different behavior of list comprehension, for, and for in a function

Question

Look at this code. I am creating 3 lists of lambda functions (stored in the variables plus_n, plus_n_, and plus_n__). They suppose to be exactly the same. However, only plus_n_ shows the expected behavior.

MAX=5
plus_n=[lambda x: x+i for i in range(MAX)]

plus_n_=[]
for i in range(MAX):
    plus_n_.append(lambda x: x+i)

def all_plus_n():
    plus_ns=[]
    for i in range(MAX):
        plus_ns.append(lambda x: x+i)
    return plus_ns
plus_n__=all_plus_n()    

for i in range(len(plus_n)):
    print('plus_n[{}]({})={}'.format(i,3,plus_n[i](3)))
    print('plus_n_[{}]({})={}'.format(i,3,plus_n_[i](3)))
    print('plus_n__[{}]({})={}'.format(i,3,plus_n__[i](3)))
    print()

The output:

plus_n[0](3)=7
plus_n_[0](3)=3
plus_n__[0](3)=7

plus_n[1](3)=7
plus_n_[1](3)=4
plus_n__[1](3)=7

plus_n[2](3)=7
plus_n_[2](3)=5
plus_n__[2](3)=7

plus_n[3](3)=7
plus_n_[3](3)=6
plus_n__[3](3)=7

plus_n[4](3)=7
plus_n_[4](3)=7
plus_n__[4](3)=7

See, the exact same code gives different results if it is on a function or in a comprehensive list...

So, what is the difference between the 3 approaches? What is happening? If I want to use this variable in multiple functions, do I have to use it as a global variable? Because seems that I cant use a function to get the variable values...

Tks in advance.

Answer 1

This is a somewhat interesting variation on the usual question. Normally, the plus_n_ version wouldn't work either, but you happen to have reused i as the iteration variable for your testing at the end of the code. Since plus_n_ captures the global i, and the test loop also sets the global i, the lambda retrieved from plus_n_ uses the correct value each time through the loop - even though it's late binding on i. The list comprehension has its own scoped i which has a value of 4 after evaluation (and doesn't change after that); similarly for the loop in the function.

---

The clean, explicit, simple way to bind function parameters in Python is functools.partial from the standard library:

from functools import partial

MAX = 5

# Or we could use `operator.add`
def add(i, x):
    return i + x

# Works as expected, whatever happens to `i` later in any scope
plus_n = [partial(add, i) for i in range(MAX)]

This way, each call to partial produces a callable object that binds its own value for i, rather than late-binding to the name i:

for j in plus_n:
    i = {"this isn't even a valid operand": 'lol'} # utterly irrelevant
    print(plus_n[j](3))

Do notice, however, that the parameters to be bound need to be at the beginning for this approach.

---

Another way to solve the specific example problem is to rely on bound method calls:

plus_n = [i.__add__ for i in range(MAX)]

---

You can also hand-roll your own currying, but why reinvent the wheel?

---

Finally, it is possible to use default parameters to lambdas to bind parameters - although I greatly dislike this method, since it is abusing the behaviour that causes another common problem and implying the existence of a parameter that could be overridden but isn't designed for it:

plus_n = [lambda x, i=i: x+i for i in range(MAX)]