Dynamic generation of partial functions in Python 2.7.x

Question

Say I want to dynamically create a function on an IPython shell from the following lambda:

f = lambda x, ci: np.percentile(x, 100-ci)

that fixes ci to a new value. It would be something like the following (create_new_f is what I am looking for).

ci = 20
new_f = create_new_f(f, ci=ci)
result = new_f([20,30,50,80])

I have tried using functools.partial as:

new_f = functools.partial(f, ci=20)

but when I run this in an embedded shell in IPython I get:

AttributeError: 'functools.partial' object has no attribute '__module__'

is there any alternative way of doing this, perhaps using decorators?

Background on the topic:

Here is an IPython issue that prompted the question.
Here is a thread explaining the problem (and fix in Python 3.x only) as well.
Here is another thread in SO that shows the problem (see the top comment)
For an even deeper analysis of this problem, see this issue in IPython.

again, what I am looking for is an alternative way of doing this.

Cannot reproduce. Please show a minimal *runnable* example that shows the error you get. — Bakuriu, Jun 02 '14 at 19:34
@Bakuriu I have updated the OP to highlight the fact that this is a problem ocurring in embedded shells. I added links to an open `BUG` that shows the problem in IPython as well. — Amelio Vazquez-Reina, Jun 02 '14 at 19:45
Instead of using `partial` you can always use a `lambda`. This should solve your problem since `lambda`s *are* real functions. — Bakuriu, Jun 02 '14 at 19:49
@Bakuriu the link to the IPython issue shows that lambdas cannot solve the problem in embedded shells either. — Amelio Vazquez-Reina, Jun 02 '14 at 19:51

score 4 · Accepted Answer · edited Jun 20 '20 at 09:12

4

You can implement with a closure (I believe this may be your solution):

import numpy as np
def partial_f(ci):
    def fn(x):
        return np.percentile(x, 100-ci)
    return fn
new_f = partial_f(20)
new_f([20,30,50,80])

returns

62.000000000000014

functools.partial works for me in my IDE's shell at work:

import functools
import numpy as np
f = lambda x, ci: np.percentile(x, 100-ci)
ci = 20
new_f = functools.partial(f, ci=ci)
new_f([20,30,50,80])

returns

62.000000000000014

With lambdas (this does work in the IPython online shell I tried it out on, but apparently doesn't work for your case):

import numpy as np
pf = lambda ci: lambda x: np.percentile(x, 100-ci)
new_f2 = pf(20)
new_f2([20,30,50,80])

also returns

62.000000000000014

edited Jun 20 '20 at 09:12

Community

answered Jun 02 '14 at 19:39

Thanks but this does not work in an IPython embedded shell (see my edits). Any other way of doing this? – Amelio Vazquez-Reina Jun 02 '14 at 19:46
@user815423426 how's that? – Russia Must Remove Putin Jun 02 '14 at 19:55
doesn't `partial_f` need to declare `x` as an input argument? – Amelio Vazquez-Reina Jun 02 '14 at 19:56
The last option does not work in embedded shells in IPython either (that was the original solution in the IPython issue) – Josh Jun 02 '14 at 20:30
The lambda definition does not pick up the definition of `ci` outside of the lambda scope. – Josh Jun 02 '14 at 20:40
I see, it won't work in the embedded IPython. I'll have to take your word for it. – Russia Must Remove Putin Jun 02 '14 at 20:49