How do variable work inside a lambda function?

Question

I am trying to connect 9 different buttons to one handler using a lambda function and some PyQt5 QPushButtons in python 3.6. If I assign them individually using ints all works fine. However if I try to use a list and a loop they are all assigned to a button number of 10. I can't understand why, since I would have thought that my assignment was to the integer value and that my variable was out of scope. Clearly, there is something I don't understand going on here. Can anyone explain the behavior of this code?

    self.buttonList = [ self.sq1Button,
                        self.sq2Button,
                        self.sq3Button,
                        self.sq4Button,
                        self.sq5Button,
                        self.sq6Button,
                        self.sq7Button,
                        self.sq8Button,
                        self.sq9Button]
    buttonNumber = 1
    for button in self.buttonList:
        button.clicked.connect(lambda: self.squareButtonHandler(buttonNumber))
        buttonNumber += 1

http://stackoverflow.com/questions/141642/what-limitations-have-closures-in-python-compared-to-language-x-closures If you look at this question it seems that typical python closures are read-only. A closure being a lambda that references a variable from outside it's scope. — Novaterata, Apr 15 '17 at 03:01
You can think of it as being similar to being an object that takes a parameter, but instead of having an explicit constructor, it's implicit with a lambda. — Novaterata, Apr 15 '17 at 03:05
@Novaterata - interesting, but they are not necessarily read-only. If some other inner function uses `nonlocal buttonNumber`, it can change the value for everybody. — tdelaney, Apr 15 '17 at 04:40
Yes, that appears to be a new addition to the language to allow read/write closures — Novaterata, Apr 15 '17 at 04:43

score 2 · Accepted Answer · answered Apr 15 '17 at 04:38

When python executes a function, it creates a namespace to hold local variables. The lambda in

button.clicked.connect(lambda: self.squareButtonHandler(buttonNumber))

is an inner function that contains a reference to buttonNumber in the outer scope. When you pass that lambda to button.clicked.connect, python has to remember that reference somehow. It does that by adding the context of the outer scope to the function object that it creates and passes to connect. The function objects for all of the buttons you connected referene the same outer context and that means they will all see whatever is in buttonNumber when the function exits.

Here is a running example showing your problem

def buttonHandler(num):
    print('button', num)

def try_lambda():
    handlers = []
    for num in range(5):
        handlers.append(lambda: buttonHandler(num))
    return handlers

print("test 1")
for handler in try_lambda():
    handler()

It produces

test 1
button 4
button 4
button 4
button 4
button 4

Yep, that's the problem. Lets take a look at the function objects we created by looking at the function object's closure

print("test 2")
for handler in try_lambda():
    handler()
    print(handler, handler.__closure__)

It shows

test 2
button 4
<function try_lambda.<locals>.<lambda> at 0x7f66e34a9d08> (<cell at 0x7f66e49fb3a8: int object at 0xa68aa0>,)
button 4
<function try_lambda.<locals>.<lambda> at 0x7f66e34a9d90> (<cell at 0x7f66e49fb3a8: int object at 0xa68aa0>,)
button 4
<function try_lambda.<locals>.<lambda> at 0x7f66e34a9e18> (<cell at 0x7f66e49fb3a8: int object at 0xa68aa0>,)
button 4
<function try_lambda.<locals>.<lambda> at 0x7f66e34a9ea0> (<cell at 0x7f66e49fb3a8: int object at 0xa68aa0>,)
button 4
<function try_lambda.<locals>.<lambda> at 0x7f66e349a048> (<cell at 0x7f66e49fb3a8: int object at 0xa68aa0>,)

Interesting. We got 4 different function objects (0x7f66e34a9d08, etc...) but a single cell holding the variable we want at 0x7f66e49fb3a8. That's why they all see the same number - they all use the same saved cell from the outer function's local variables.

In your case, partial is a better option. It creates a function using a variable's current value and works likes you want.

import functools

def try_partial():
    handlers = []
    for num in range(5):
        handlers.append(functools.partial(buttonHandler, num))
    return handlers

print("test 3")
for handler in try_partial():
    handler()

It produces

test 3
button 0
button 1
button 2
button 3
button 4

An option without `partial` is to make `num` local with `handlers.append(lambda num=num: buttonHandler(num))`. — Mark Tolonen, Apr 17 '17 at 03:43
I had used this once before. A friend gave me the code, but I don't think I really understood it fully. Now thanks to tdelaney, I believe I do! Thanks! However, when I tried to keep the lambda by creating a local variable, it only worked if I used (lambda checked, num=num: buttonHandler(num)). I copied this from the code I was given, but I'm clueless as to what the "checked," bit is doing... — Arana, Apr 26 '17 at 22:54

score 1 · Answer 2 · answered Apr 15 '17 at 04:30

I had the same problem once and this helped me. What you basically need to do is move the click handler to a separate function and call the function with the buttonNumber from inside the loop. This is probably due to how closures work and/or because it needs a new buttonNumber every time the loop runs. I still don't understand the exact reason so if anyone does, please comment/edit.

How do variable work inside a lambda function?

2 Answers2