0

I am learning lambdas in Python. I need to create a list of functions f = [f1, f2...] such that each function fi(x) takes in a list x and returns (x[i]-1).

This is how I tried coding it, but I am getting surprising results. Please help me understand why each of the three prints give different results. The last two have left me absolutely stumped!

f = [(lambda x: x[i]-1) for i in range(5)]
# I expect f to be [ (lambda x: x[0]-1), (lambda x: x[1]-1), ...]

x = [1, 2, 3, 4, 5]

print f[0](x), f[1](x), f[2](x)    # output: 4 4 4 !!
print [f[i](x) for i in range(5)]  # output: [0, 1, 2, 3, 4] as expected
print [f[k](x) for k in range(5)]  # output: [4, 4, 4, 4, 4] wha?!!!

Edit: This question is quite different from the suggested duplicate. In the linked question there was a simple error where the user created a list of functions, instead of function-calls. However the answer by Tomasz Gandor discusses the same issue as asked here using several examples.

hynekcer
  • 14,942
  • 6
  • 61
  • 99
Neha Karanjkar
  • 3,390
  • 2
  • 29
  • 48
  • https://stackoverflow.com/questions/6076270/python-lambda-function-in-list-comprehensions – 101 Nov 08 '17 at 10:31
  • Just out of curiosity, why are you coding something so .. weird? Pure academic interest? After all, Guido regrets having implemented them: http://legacy.python.org/doc/essays/ppt/regrets/PythonRegrets.pdf – Arne Nov 08 '17 at 10:36
  • well in Python 3 it doesn't work with `i` either... – Szabolcs Nov 08 '17 at 10:40
  • The only regrettable thing about Python lambdas is that they're not implemented *better*. – BrianO Nov 08 '17 at 10:42
  • @Arne Recknagel: An optimization method (scipy.cobyla) requires constraints to be specified as a list of functions, one for each dimension. I need to convert from simple bounds to constraints, leaving the number of dimensions a variable. – Neha Karanjkar Nov 08 '17 at 12:08
  • @NehaKaranjkar Fair enough. Good luck! – Arne Nov 08 '17 at 14:21

2 Answers2

6

That's a common gotcha. What you're looking for is

f = [(lambda x, i=i: x[i]-1) for i in range(5)]
z = [1,2,3,4,5]
print f[0](z),f[1](z),f[2](z)
0 1 2

Your list comprehension creates 5 functions, which all points to i. At the end of the loop, i equals 4. Hence no matter what lambda you call, it evalutes i as 4.

You want to assign the current value of i, to the local variable of the lambda function. You do this with optional arguments.

Lambda evalutes params on runtime. Python will allow you to define something such this, it might shed some more light:

>>> f = lambda x: x[i] - 1
>>> l = [1, 2, 3, 4]
>>> f(l)

Traceback (most recent call last):
  File "<pyshell#91>", line 1, in <module>
    f(l)
  File "<pyshell#89>", line 1, in <lambda>
    f = lambda x: x[i] - 1
IndexError: list index out of range

Update: replying to questions from comments, this example is less related to the original question but helps to better understand how function params evaluation happens

myvar=1

def f(myvar=myvar):
    print myvar

f()  # prints 1
f(2) # prints 2
myvar = 5
f()  # prints 1

If you don't declare myvar before declaring f, you would get NameError: name 'myvar' is not defined. This because function variables evaluted on 'compile' time. Lambda's evalutes on runtime, but when we provide optional arguments - it happens immediately (hence i=i). Hope it provides more clarity on the subject :)

Chen A.
  • 10,140
  • 3
  • 42
  • 61
  • Just to clarify, wouldn't it be a better idea to define the index with a different name in order to avoid giving a parameter the same name as its value? E.g.: `f = [(lambda x, idx=i: x[idx]-1) for i in range(5)]` – Arne Nov 08 '17 at 14:24
  • Python's scoping rules are not intuitive to me (coming from a C/C++ background). In my example, I had assumed that the scope of i was limited to the first line. – Neha Karanjkar Nov 08 '17 at 14:27
  • @ArneRecknagel it's a matter of choice, since you don't address it anymore in your code besides your lambda definition - both options are clear enough. – Chen A. Nov 08 '17 at 14:31
  • @NehaKaranjkar it's not a matter of scoping, but how lambda's work. See my snippet how lambda is evaluted at runtime, and not at declaration :-) – Chen A. Nov 08 '17 at 14:32
  • @Vinny I assumed that it behaves exactly as a function head. Say, `my_var = 1; def my_func(my_var=my_var): print(my_var) # prints '1'` , the assignment of the parameter to the value is completely valid, but I would avoid giving them the same name, simply to avoid the confusion of an, intuitively, nonsensical line. Am I wrong in understanding how lambdas work? – Arne Nov 08 '17 at 15:01
  • @ArneRecknagel functions evalute signature parameters on definition. Your understanding is correct, but if you change the value of `my_var` afterwards your function would keep printing the original value. Why is that? because you assigned the value of `my_var` when python interpreted your `def` line. I've updated my answer with an example of how it is interpreted. – Chen A. Nov 08 '17 at 15:27
3

The variable i is in global scope and is still 4 when you call any of the functions. If you do i = 1 then your two anomalous outputs will become 1 1 1 and [1 1 1 1 1]

I found this works:

g = lambda i: lambda x: x[i] - 1
f = [ g(i) for i in range(5) ]  # i missed here
print [f[k]([1,2,3,4,5]) for k in range(5)] # [0, 1, 2, 3, 4]

The i is now scoped to a second lambda.

Szabolcs
  • 3,990
  • 18
  • 38
HP Williams
  • 159
  • 2