How to create a set of parametrized functions in python?

Question

I am trying to create a family of functions, parametrized by a "shift" parameter. Consider the code below.

I want the for loop to create a set of functions, each of which shifts the argument by a different amount.

# Python 3 code.
N = 5
fns = []
for idx in range(N):
    def f(x): # shift input by idx
        return x+idx
    fns.append(f)


print("---------------------------")
for i in range(N):
  print(fns[i](1))



print("---------------------------")
for idx in range(N):
  print(fns[idx](1))

After I run the code, I get something strange:

---------------------------
5
5
5
5
5
---------------------------
1
2
3
4
5

Why is the output from the two for loops different although they are doing conceptually the same thing?

Possible reason?: In the 2nd loop, I have reverted back to the name idx which is the one used in the first for loop in the creation of the list of functions.

So probably there is some "binding" magic going on here i.e. all functions basically use the value of idx in the immediate surrounding scope during its invocation.

In the 2nd for loop the value of idx at the end of the first for loop is 4, which causes the 2nd for loop to output identical results. In the 3rd for loop, idx variable is again changing value at each iteration which causes the functions to behave as intended in this 3rd for loop.,

Is this a correct explanation?

But this is clearly a buggy way of defining a one-parameter family of functions. What then is the Pythonic way of doing this?

Note: Some might suggest that defining the f's in the first loop as accepting two arguments: the actual argument, and the shift parameter. In my code, for certain technical reasons I cannot do this in the code-base I am working with. Maybe currying is an option? I don't want to go there, if there is a simpler Python solution to creating a one(two,three...) parameter family of functions

Yes, in your function, `idx` is a free variable, it resolves to the global variable `idx`, hence the behavior, because in your second loop, your loop target is `idx`, so that variable keeps getting re-assigned. It's the same thing as `idx = 99; print(fns[0](2))` — juanpa.arrivillaga, Jan 17 '21 at 07:25
The correct way to do this is to write a factory function, basicalyl, you need to create an enclosing scope that where that variable isn't being changed. I prefer something like `def func_maker(idx): def f(x): return x + idx; return f` (of course, with correct indenation) and then use `fns.append(func_maker(idx))` in a loop. — juanpa.arrivillaga, Jan 17 '21 at 07:27
While the root cause of the linked question and this question is same, I think the difference in usage of lambda function and normal function in both makes it different and it should not be closed as duplicate. Leaving it on community whether we should re-open the question *(don't want to hammer back, because I am unsure about it)* — Moinuddin Quadri, Jan 17 '21 at 07:40

Moinuddin Quadri · Accepted Answer · 2021-01-17T07:58:18.217

The difference in output in both the loops in because of the difference in the value of idx.

In your first for loop, you stored the reference of function f() in fns list. Here, value of idx is not stored as part of your function f() inside fnx.

for idx in range(N):
    def f(x): # shift input by idx
        return x+idx
    fns.append(f)

Once you executed this loop, idx is holding the value as 4 (N-1 from the for loop).

Hence, your second loop print the values using idx as 4. i.e. f() printed the value 4+1

for i in range(N):
  print(fns[i](1))

However in your 3rd loop, you are re-initializing the the value of idx as part of for loop and your fns is getting the value in range 0 to 4 with each iteration.

for idx in range(N):
  print(fns[idx](1))

That's why you see the value ranging from 1 to N here (because of idx ranging from 0 to N-1).

Whew! Thanks! The code was a simplified version of a chunk in my code-base that had been causing a nasty issues for the last 2 days. — smilingbuddha, Jan 17 '21 at 07:36

How to create a set of parametrized functions in python?

1 Answers1

Linked