0

I'm using list expansion to create a set of lambda functions. Each lambda function returns a generator with an initial value. The API I am using requires that I pass it a 0-parameter function handle that returns a generator object. Hence the odd form of lambda: generator.

I get unexpected results, shown below. Can anyone explain why I only see the generator seeded with 200 when using list expansion?

All code is copy/paste runnable.

Case 1: Using list expansion

def mygen(val):
  for i in range(3):
    yield val + i

generators = [lambda: mygen(start) for start in [100, 200]]

g = generators[0]()                  # generators[1]() produces the same result

for _ in range(3):
  print(next(g))

Result:

200
201
202

Expected Result:

100
201
202

Case 2: Without list expansion

def mygen(val):
  for i in range(3):
    yield val + i

g0 = lambda: mygen(100)
g1 = lambda: mygen(200)

g = g0()

for _ in range(3):
  print(next(g))

Result (as expected)

100
101
102
David Parks
  • 30,789
  • 47
  • 185
  • 328
  • 1
    I'm fairly certain the issue is because the last item in the list `[100, 200]`, i.e. `200` is over-writing the argument passed to the function `mygen` at module runtime (before invocation of `mygen`). This is because default argument values are defined when `def` is evaluated at module run time (once) and not at invocation (after module runtime). So, when you execute the function `mygen()`, the argument `val` is defined as the last latest value given during the `def` evaluation. Did I make it understandable? – Tobiah Rex Jan 28 '18 at 23:04
  • 3
    @TobiahRex that's not exactly what is happening. Fundamentally, Python has lexical scoping with late-binding behavior. There is no default-arguments being used here, indeed, the common hack to simulate early binding in Python lambdas defined in a for-loop is to use default arguments, e.g. `[lambda start=start: mygen(start) for start in [100, 200]]`, The "proper" less hacky way is to create a new scope, nesting in another function: `[(lambda x: lambda: mygen(x))(start) for start in [100, 200]]`, although, it's a little more verbose. – juanpa.arrivillaga Jan 28 '18 at 23:10
  • Oh, that's very helpful @juanpa.arrivillaga, I hadn't picked up the simple use of default arguments in the lambda function from the duplicate question. I really appreciate the comment. From both of you! – David Parks Jan 28 '18 at 23:15
  • 1
    @DavidParks my preferred solution is to simply define a factory function, e.g. `def factory(start): return lambda: mygen(start)` then `gs = [factory(start) for start in some_list]` – juanpa.arrivillaga Jan 28 '18 at 23:18
  • @juanpa.arrivillaga Nice observation!! I made a mistake verbally. I should have said *initial argument* not *default argument*. I agree it's an important distinction. Thanks for clarifying. I am curious, I've never heard the words **lexical scoping** regarding Python. Would you say that maps to the *Local Scope* from Pythons LEGB (Local, Enclosing, Global, Builtin) or another? – Tobiah Rex Jan 28 '18 at 23:18
  • 1
    @TobiahRex no, rather, there are two different approaches to scoping, although I would say most people are familiar with lexical scoping. Not enough to go into it in a comment, but [the wikipedia article](https://en.wikipedia.org/wiki/Scope_(computer_science)#Lexical_scope_vs._dynamic_scope) covers it well. – juanpa.arrivillaga Jan 28 '18 at 23:20
  • 1
    @TobiahRex but briefly, in lexical scoping (i.e.what Python uses) `x = 10; def f():print(x); def g(): x = 42; f()` then `g()` will print `10`, if Python had dynamic scoping, it would print `42` ... as you might imagine, that could get messy... – juanpa.arrivillaga Jan 28 '18 at 23:22

0 Answers0