lambda operators in python loops

Question

I'm encountering some strange behavior with lambda functions in a loop in python. When I try to assign lambda functions to dictionary entries in a list, and when other entries in the dictionary are used in the function, only the last time through the loop is the lambda operator evaluated. So all of the functions end up having the same value!

Below is stripped-down code that captures just the parts of what I'm trying that is behaving oddly. My actual code is more complex, not as trivial as this, so I'm looking for an explanation and, preferably, a workaround.

n=4
numbers=range(n)
entries = [dict() for x in numbers]

for number, entry in zip(numbers,entries):
    n = number
    entry["number"] = n
    entry["number2"] = lambda x: n*1

for number in numbers:
    print(entries[number]["number"], entries[number]["number2"](2))

The output is:

In other words, the dictionary entires that are just integers are fine, and were filled properly by the loop. But the lambda functions — which are trivial and should just return the same value as the "number" entries — are all set to the last pass through.

What's going on?

score 2 · Answer 1 · edited May 23 '17 at 10:27

By the end of your for loop, the n variable - which, unlike in static languages such as C#, is set to 3, which is then being accessed in the lambda expression. The variable value is not fixed; as another answer on the site points out, lambda expressions are fluid and will retain references to the variables involved instead of capturing the values at the time of creation. This question also discusses your issue.

To fix it, you need to give the lambdas new, local variable via default parameters:

entry["number2"] = lambda x, n=n: n*1

This creates a new variable in the lambda's scope, called n, which sets its default value to the "outside" value of n. Note that this is the solution endorsed by the official FAQ, as this answer by Adrien Plisson states.

Now, you can call your lambda like normal and ignore the optional parameter, with no ill effect.

EDIT: As originally stated by Sci Prog, this solution makes n = number redundant. Your final code will look similar to this:

lim = 4
numbers = range(lim)
entries = [dict() for x in numbers]

for number, entry in zip(numbers, entries):
    entry["number"] = number
    entry["number2"] = lambda x, n = number: n*1

for number in numbers:
    print(entries[number]["number"], entries[number]["number2"](2))

good answer, although I would remove the comment: "# Optional: del lim unless it will be necessary later". I don't see the need to unbind the `lim` reference from the global namespace here. Except for some oddball cases, `del` is generally considered bad style and not used. For a trivial script like this, unbinding a single reference to an integer would be unconventional and unpythonic. — Corey Goldberg, Mar 16 '16 at 18:02
Not gonna pretend to know anything about the high-level python conventions. most of my python knowledge is as a hobbyist. Thanks. — MutantOctopus, Mar 16 '16 at 19:51

Sci Prog · Answer 2 · 2016-03-16T02:28:46.233

2

Try this

N=4
numbers=range(N)
entries = [dict() for x in numbers]

for number, entry in zip(numbers,entries):
    entry["number"] = number
    entry["number2"] = lambda x,n=number: n*1

for number in numbers:
    print(entries[number]["number"], entries[number]["number2"](2))

It prints (python3)

To avoid confusion, n referred to different things in your code. I used it only at one place.

It is a closure problem.

edited Mar 16 '16 at 02:28

answered Mar 16 '16 at 02:12

Sci Prog

2,651
1
10
18

You also got an answer for BHustus while I was typing mine. I upvoted his answer. – Sci Prog Mar 16 '16 at 02:15
+1 for pointing out the redundancy of still using n in this solution. Mind if I rework my answer to reflect it? – MutantOctopus Mar 16 '16 at 02:17
No problem, go ahead. It improves clarity. – Sci Prog Mar 16 '16 at 02:18
Thanks. If I infringed too much on you feel free to tell me and I'll scale it back. :) – MutantOctopus Mar 16 '16 at 02:21

Farmer Joe · Answer 3 · 2016-03-16T02:16:55.410

You are probably reaching the problem that the method is created as referencing a variable n. The function is only evaluated after the loop so you are going to call the function which references n. If you're ok with having the function evaluated at the time of assignment you could put a function call around it:

 (lambda x: n*1)(2)

or if you want to have the functions to use, have them reference the specific value you want. From your code you could use a default argument as a workaround:

entry["number"] = n
entry["number2"] = lambda x, n=n: n*1

The difference comes down to a question of memory addressing. I imagine it went something like this:

You:     Python, please give me a variable called "n"
Python:  Ok!  Here it is, it is at memory slot 1
You:     Cool!  I will now create functions which say take that variable "n"
         value (at memory slot 1) and multiply it by 1 and return that to me.
Python:  Ok! Got it:
             1. Take the value at memory slot 1.
             2. Multiply by 1.
             3. Return it to you.
You:     Done with my looping, now evaluate those instructions!
Python:  Ok!  Now I will take the value of at memory slot 1 and multiply by 1 
         and give that to you.
You:     Hey, I wanted each function to reference different values!
Python:  I followed your instructions exactly!

lambda operators in python loops

3 Answers3