2

Problem:

I want to create a list of lambda functions using **kwargs which are iterated in the list.

Similar questions (e.g., e.g.) exist, but they do not consider iterating over **kwargs.

The problem, as before, is that the value of kwargs in the lambda function is evaluated "lazily" after creation in the list, meaning the last value assigned during the iteration is passed to all lambda functions. I've verified this is a 'problem' for classic loops as well as comprehensions.

(2a) is not, having the value argset from the last iteration of the comprehension assigned to all **kwargs. (3a) is worse, having both the values of argset and i from the last iteration assigned to x and **kwargs.

MWE

Code

def strfun(x,**kwargs):
  return 'x: {} | kwargs: {}'.format(x,kwargs)

argsets = [
  {'foo': 'bar'},
  {'baz': 'qux'},
]

# (1) expected behaviour:
print '(1) '+str([strfun(i,**argset) for i,argset in enumerate(argsets)])

# (2) unexpected behaviour:
funs = [lambda x: strfun(x,**argset) for argset in argsets]
print '(2) '+str([fun(i) for i,fun in enumerate(funs)])

# (3) unexpected behaviour:
funs = [lambda : strfun(i,**argset) for i,argset in enumerate(argsets)]
print '(3) '+str([fun() for fun in funs])

Output:

(1) ["x: 0 | kwargs: {'foo': 'bar'}", "x: 1 | kwargs: {'baz': 'qux'}"]
(2) ["x: 0 | kwargs: {'baz': 'qux'}", "x: 1 | kwargs: {'baz': 'qux'}"]
(3) ["x: 1 | kwargs: {'baz': 'qux'}", "x: 1 | kwargs: {'baz': 'qux'}"]

(1) is "correct".

(2) is not, having the last value of argsets assigned to **kwargs for all functions ({'baz': 'qux'}).

(3) is worse, having the last value of both i and argsets assigned to x and **kwargs for all functions (1 and {'baz': 'qux'}).

jessexknight
  • 756
  • 7
  • 20
  • 1
    I'm pretty sure this is the same issue described [here](https://stackoverflow.com/questions/12423614/local-variables-in-python-nested-functions). The `**kwargs` syntax doesn't make any difference to this issue. Nor does the fact that you're using `lambda` to make your functions in a comprehension rather than an explicit `def` statement in a normal loop. – Blckknght Aug 01 '18 at 16:04
  • Yes it seems it is the same issue, though I don't think that's obvious to a python beginner. In my case, both explicit keyword scoping and new function scoping are not feasible since the functions are not known before hand (to this module). However, the partial solution is viable. I'll update the solution. – jessexknight Aug 01 '18 at 16:32
  • I'm pretty sure you can make the default argument solution work too. `[lambda x, kwargs=kwargs: fun(x, **kwargs) for kwargs in kwargset]` – Blckknght Aug 01 '18 at 16:48
  • Re. closing this question: I had already included links to other similar-but-different questions, and described why my question was unique before the question was closed ... let's reopen please. – jessexknight Aug 17 '22 at 12:59

1 Answers1

3

Solutions

Solution 1: functools.partial

As suggested by the solution linked by @Blckknght in the comment above, functools.partial is probably the cleanest way to do this (see below).

Solution 2: nested lambda

As suggested by @jfs in this answer, a workaround is to define an outer layer of lambda to force evaluation of the current value of the iterated object during assignment of the inner lambda, and roll over the outer lambda using map, creating the desired list.

Very minimal code:

change

[lambda x: fun(x,**kwargs) for kwargs in kwargset]

to (1)

[partial(fun, **kwargs) for kwargs in kwargset]

or (2)

map(lambda kwargs: (lambda x: fun(x,**kwargs)), kwargset)

Full MWE

Code

from functools import partial

def strfun(x,**kwargs):
  return 'x: {} | kwargs: {}'.format(x,kwargs)

argsets = [
  {'foo': 'bar'},
  {'baz': 'qux'},
]
# (1) always expected behaviour:
print '(1)   '+str([strfun(i,**argset) for i,argset in enumerate(argsets)])

# (2)
# unexpected behaviour:
funs = [lambda x: strfun(x,**argset) for argset in argsets]
print '(2-x) '+str([fun(i) for i,fun in enumerate(funs)])
# expected behaviour
funs = map(lambda argset: (lambda x: strfun(x,**argset)), argsets)
print '(2-1) '+str([fun(i) for i,fun in enumerate(funs)])
# expected behaviour
funs = [partial(strfun, **argset) for argset in argsets]
print '(2-2) '+str([fun(i) for i,fun in enumerate(funs)])

# (3)
# unexpected behaviour:
funs = [lambda : strfun(i,**argset) for i,argset in enumerate(argsets)]
print '(3-x) '+str([fun() for fun in funs])
# expected behaviour
funs = map(lambda (i,argset): (lambda : strfun(i,**argset)), enumerate(argsets))
print '(2-1) '+str([fun() for fun in funs])
# expected behaviour
funs = [partial(strfun, i, **argset) for i,argset in enumerate(argsets)]
print '(2-2) '+str([fun() for fun in funs])

Output:

(1)   ["x: 0 | kwargs: {'foo': 'bar'}", "x: 1 | kwargs: {'baz': 'qux'}"]
(2-x) ["x: 0 | kwargs: {'baz': 'qux'}", "x: 1 | kwargs: {'baz': 'qux'}"]
(2-1) ["x: 0 | kwargs: {'foo': 'bar'}", "x: 1 | kwargs: {'baz': 'qux'}"]
(2-2) ["x: 0 | kwargs: {'foo': 'bar'}", "x: 1 | kwargs: {'baz': 'qux'}"]
(3-x) ["x: 1 | kwargs: {'baz': 'qux'}", "x: 1 | kwargs: {'baz': 'qux'}"]
(2-1) ["x: 0 | kwargs: {'foo': 'bar'}", "x: 1 | kwargs: {'baz': 'qux'}"]
(2-2) ["x: 0 | kwargs: {'foo': 'bar'}", "x: 1 | kwargs: {'baz': 'qux'}"]

(2-1),(2-2),(3-1),(3-2) illustrate the possible workarounds.

jessexknight
  • 756
  • 7
  • 20