How to create consistent lambda functions using a loop?

Question

I want to create several similar lambda functions.

When I use a simple loop I do not get what I expect.

Code:

funcList = []
for i in range(2):
    f = lambda x: i*x
    print("Before: ", f(10))
    funcList.append(f)

print("After: ", funcList[0](10), funcList[1](10))

Output:

Before:  0
Before:  10
After:  10 10

Code:

funcList = [lambda x: i*x for i in range(2)]
print("After: ", funcList[0](10), funcList[1](10))

Output:

After:  10 10

How can I create several lambda functions, with it using the "original" value of i, instead of the last known value of i?

Answer 1

The issue is as follows: closures (see lambdas vs closures) in Python close over their environment, which maps variable names to objects, and they do this by essentially adding the closed-over environment to their scope lookup chain. Consider a simpler, more explicit example:

import sys
def foo():
    result = []
    for i in range(2):
        def bar():
            return i
        result.append(bar)
    return result

The scope lookup chain for foo, with the environment right before it returns, is something like:

- "foo" local variables: {i: 1, result: [bar-function-1, bar-function-2], bar: bar-function-2}
- global variables: {foo: foo-function, sys: sys-module, etc...}

That is, if foo tries to use a variable z, first it looks in the first environment (foo's local variables). If it is there, the lookup succeeds. If not, it moves onto the the next (the global variables). If it is there, the lookup succeeds. If not, then there are no more environments on the chain so you get a NameError.

In this case, result and i would be found as local variables, foo and sys and perhaps others would be found as global variables, and all else would give a NameError.

The scope lookup chain in each bar is something like:

- "bar" local variables: {}
- "foo" local variables: {i: 1, result: [bar-function-1, bar-function-2], bar: bar-function-2}
- global variables: {foo: foo-function, sys: sys-module, etc...}

Most importantly, foo's local variable environment is not copied into bar's local variable environment. So, bar can lookup i, but it does this by first failing to find it in bar's local variables, then going one up the scope chain and finding it in foo's local variables. So, when the first bar function is defined, its scope lookup chain looks like this:

- "bar" local variables: {}
- "foo" local variables: {i: 0, result: [], bar: bar-function-1}
- global variables: {foo: foo-function, sys: sys-module, etc...}

However, when foo changes its local variable i, bar's scope lookup chain now looks like this:

- "bar" local variables: {}
- "foo" local variables: {i: 1, result: [bar-function-1], bar: bar-function-2}
- global variables: {foo: foo-function, sys: sys-module, etc...}

So now when bar looks up i it yet again fails to find it in its local variables, looks one up the scope lookup chain, and finds foo's local variable i... which is now 1 since it's the same i as before.

---

The trick I wrote in the comment is a bit of a hack. To be a bit more explicit about it, consider:

def foo():
    result = []
    for i in range(2):
        def bar(j=i):
            return j
        result.append(bar)
    return result

What's really happening is that you're declaring bar with a parameter, j, whose default is set to the value of i (i.e. the object which i is referring to at definition time... not the object which i refers to at any time j is used inside of bar). So the scope lookup chain at the first bar function looks like this:

- "bar" local variables: {j: 0}
- "foo" local variables: {i: 0, result: [], bar: bar-function-1}
- global variables: {foo: foo-function, sys: sys-module, etc...}

And by the time the loop goes around again, it looks like this:

- "bar" local variables: {j: 0}
- "foo" local variables: {i: 1, result: [bar-function-1], bar: bar-function-2}
- global variables: {foo: foo-function, sys: sys-module, etc...}

In both cases, the lookup of j succeeds right away in bar's local variables, and j never changes.

---

Doing the following is a bit of a hack in that it just hides the outer i and makes it look like it's referring to the same i:

def foo():
    result = []
    for i in range(2):
        def bar(i=i):
            return i
        result.append(bar)
    return result

But in reality, they are two different is:

- "bar" local variables: {i: 0}
- "foo" local variables: {i: 0, result: [], bar: bar-function-1}
- global variables: {foo: foo-function, sys: sys-module, etc...}

And, on the second iteration of the loop:

- "bar" local variables: {i: 0}
- "foo" local variables: {i: 1, result: [bar-function-1], bar: bar-function-2}
- global variables: {foo: foo-function, sys: sys-module, etc...}

---

Probably a more "proper" way would be to do something like:

def make_bar(j):
    def bar():
        return j
    return bar

def foo():
    result = []
    for i in range(2):
        bar = make_bar(i)
        result.append(bar)
    return result

In this case the scope chain is:

- "bar" local variables: {}
- "make_bar" local variables: {j: 0}
- "foo" local variables: {i: 0, result: [], bar: bar-function-1}
- global variables: {make_bar: make_bar-function, foo: foo-function, sys: sys-module, etc...}

And, on the second iteration of the loop:

- "bar" local variables: {}
- "make_bar" local variables: {j: 0}
- "foo" local variables: {i: 1, result: [], bar: bar-function-2}
- global variables: {make_bar: make_bar-function, foo: foo-function, sys: sys-module, etc...}

In this case it works because make_bar is called with i, make_bar is called setting its local variable j to the object that i refers to at call-time (namely, 0). make_bar's local variable j doesn't change when foo's i changes.

---

To do a completely explicit example in your case you'd do something like:

funcList = []
for i in range(2):
    def make_f(j):
        def f(x):
            return j * x
    funcList.append(make_f(i))

Or, as @ShadowRanger commented:

funcList = []
for i in range(2):
    funcList.append((lambda j: lambda x: j*x)(i))

Or just:

 funcList = [(lambda j: lambda x: j*x)(i) for i in range(2)]

Or you can use the hack I proposed, now knowing the full story of why it works:

 funcList = [lambda x, i=i: i*x for i in range(2)]

Answer 2

This is due to late binding.

1. You have created a list of 2 functions
2. Whats are these functions? – fun(x): return i*x
3. At this stage python doesn’t care about value of i or x
4. But what is the value of i at the end of loop? ‘i’ should be 1
5. When you call funcList[0](10), it will return i*x
6. Here x is 10 and i is 1, So all the call to funcList[0](10) will return 10

Answer already given in comment f = lambda x,i=i: i*x, It will avoid late binding, value of i would be resolved while creating function object.