How to specify arg position for functool partial()

Question

As per manual, functools partial() is 'used for partial function application which “freezes” some portion of a function’s arguments and/or keywords resulting in a new object with a simplified signature.'

What's the best way to specify the positions of the arguments that one wishes to evaluate?

EDIT

Note as per comments, the function to be partially evaluated may contain named and unnamed arguments (these functions should be completely arbitrary and may be preexisting)

END EDIT

For example, consider:

def f(x,y,z):
    return x + 2*y + 3*z

Then, using

from functools import partial

both

partial(f,4)(5,6)

and

partial(f,4,5)(6)

give 32.

But what if one wants to evaluate, say the third argument z or the first and third arguments x, and z?

Is there a convenient way to pass the position information to partial, using a decorator or a dict whose keys are the desired arg positions and the respective values are the arg values? eg to pass the x and z positions something like like this:

partial_dict(f,{0:4,2:6})(5)

sounds like a XY problem. Use keyword, not positional in that case. Third argument is easy: just use `partial(f,z=4)` and pass x, y params as positional when calling. — Jean-François Fabre, Mar 02 '20 at 21:08
@Jean-FrançoisFabre, no the arguments are not named - look at the definition of `f` in the example, and cannot assumed that they are named as the functions can be preexisting. Will edit Q to clarify. — alancalvitti, Mar 02 '20 at 21:41
@alancalvitti Could you clarify what you mean by "unnamed argument"? Do you mean C API defined functions with no parameter names, [positional-only](https://www.python.org/dev/peps/pep-0570/) parameters, or variadic `*args` style parameters? Except for these special cases, all conventionally defined functions in Python can be called using keywords arguments in place of positional arguments. — Brian61354270, Mar 02 '20 at 21:53
@Brian, no parameter names, just like `f` in the example above. I don't think it would work well with `*arg` style parameters, but it should also work with named parameters. — alancalvitti, Mar 02 '20 at 21:56
@alancalvitti I see three parameter names, `x`, `y`, and `z`... — Brian61354270, Mar 02 '20 at 21:57
@Brian, I thought you were referring to `**kwarg` style named parameters, like `base` in `int('73', base=10)` — alancalvitti, Mar 02 '20 at 22:00
@alancalvitti [This answer](https://stackoverflow.com/a/1419160/11082165) to a related question may help clarify the difference between `f(z=10` "keyword arguments" and `def f(**kwargs)`, which takes arbitrary keyword arguments as a separate parameter. As a side note, the builtin `int` *does not* accept arbitrary `**kwargs`, but rather has a parameter named `base` that happens to have a default value. E.g., give `int('73', 8)` a try. — Brian61354270, Mar 02 '20 at 22:05
@Brian, thanks, I'm looking for a general solution that will work with named arguments (like `f(x,y,z)` in the example), with keyword arguments and with arbitrary `**kwargs` if these make sense in this context. — alancalvitti, Mar 02 '20 at 23:20

blhsing · Accepted Answer · 2020-03-03T16:46:37.623

4

No, partial is not designed to freeze positional arguments at non-sequential positions.

To achieve the desired behavior outlined in your question, you would have to come up with a wrapper function of your own like this:

def partial_positionals(func, positionals, **keywords):
    def wrapper(*args, **kwargs):
        arg = iter(args)
        return func(*(positionals[i] if i in positionals else next(arg)
            for i in range(len(args) + len(positionals))), **{**keywords, **kwargs})
    return wrapper

so that:

def f(x, y, z):
    return x + 2 * y + 3 * z

print(partial_positionals(f, {0: 4, 2: 6})(5))

outputs:

edited Mar 03 '20 at 16:46

answered Mar 02 '20 at 22:18

blhsing

91,368
6
71
106

I've updated the answer to allow default keyword arguments to be specified for the custom `partial` function, as suggested by your comment. – blhsing Mar 03 '20 at 16:47
Thanks, I've accepted this. But I'd also like to ask if you think there are more syntactically convenient method to specify the arguments being held, ie, not evaluated. For example in another language (Mathematica), you can use literally a bullet point (option-8) like this `f(x,•,z)` to turn that function into the partially evaluated `f(x,z)`. I don't think this is feasible in python but maybe there's similar approach? – alancalvitti Mar 11 '20 at 16:09
You're welcome. I don't think there's anything built-in that's close to that usage in Python. What you want to achieve is typically and arguably more Pythonically done by specifying keyword arguments for the `partial` function instead. Positional arguments are meant to be specified sequentially in Python. – blhsing Mar 11 '20 at 16:17
I've added a new answer to tailor the solution to the Mathematica style of partial argument specifications. – blhsing Mar 11 '20 at 16:50
Re "What you want is... more Pythonically done by specifying keyword arguments". No, that's not my intention. I think I need to rephrase the question. I am looking for a general partial-eval solution for any function including pre-existing ones, which may or may not have keyword args, and even if they do, you typically want to hold a non-keyword arg. – alancalvitti Mar 12 '20 at 13:44
Yes I know what you're looking for, but I was just saying that Python coders typically work around the limitation by simply switching to keyword arguments when needing to skip sequentially positioned arguments. – blhsing Mar 12 '20 at 13:55
keyword args don't address the partial evaluation issue in any way, whether the keywords are named or `**kwarg`. Partial evaluation means the function has to be turned into a lambda applicable to the selected held (ie, not evaluated) args. If a function is defined with `f(x=1,y=2,z=3)`, calling it with `f(x=10,z=30)` simply invokes the default value `y=2` rather than converting `f` into a lambda. – alancalvitti Mar 12 '20 at 14:43
Yes I know what you mean. I was just saying that typically arguments are named in Python so people can get around the issue by partially specifying argument values by names instead. Anyway, does my other answer better emulate what you were looking for syntactically after all? – blhsing Mar 12 '20 at 17:34

Brian61354270 · Answer 2 · 2020-03-02T21:35:14.550

Simply use keyword arguments. Using your definition of f above,

>>> g = partial(f, z=10)
>>> g(2, 4)
40
>>> h = partial(f, y=4, z=10)
>>> h(2)
40

Note that once you use a keyword argument for a given parameter, you must use keyword arguments for all remaining arguments. For example, the following would not be valid:

>>> j = partial(f, x=2, z=10)
>>> j(4)
TypeError: f() got multiple values for argument 'x'

But continuing to use keyword arguments is:

>>> j = partial(f, x=2, z=10)
>>> j(y=4)
40

When you use functools.partial, you store the values of *args and **kwargs for later interpolation. When you later call the "partially applied" function, the implementation of functools.partial effectively adds the previously provided *args and **kwargs to the argument list at the front and end, respectively, as though you had inserted these argument-unpackings yourself. I.e., calling

h = partial(1, z=10)
f(4)

is roughly equivalent to writing

args = [1]
kwargs = {'z': 10}
f(*args, 4, **kwargs)

As such, the semantics of how you provide arguments to functools.partial is the same as how you would need to store arguments in the args and kwargs variables above such that the final call to f is sensible. For more information, take a look at the pseduo-implementation of functools.partial given in the functools module documentation

I'm looking for a general solution whether named or unnamed arguments - see my comments to Jean-Francois and subsequent clarification in my question. — alancalvitti, Mar 02 '20 at 21:44
@alancalvitti The second half of my answer addresses the general case. Everything you pass to `functools.partial` will behave as though you stored and unpacked the arguments manually in the style provided above. If you cannot use keyword arguments (due to one of the three reasons I mentioned in my comment on your question above), you will likely need to define a wrapper function which you can partially evaluate. — Brian61354270, Mar 02 '20 at 21:58

blhsing · Answer 3 · 2020-03-11T17:00:49.880

For easier usage, you can create a new object specifically to specify a positional argument that is to be skipped when sequentially listing values for positional arguments to be frozen with partial:

SKIP = object()

def partial_positionals(func, *positionals, **keywords):
    def wrapper(*args, **kwargs):
        arg = iter(args)
        return func(*(*(next(arg) if i is SKIP else i for i in positionals), *arg),
            **{**keywords, **kwargs})
    return wrapper

so that:

def f(x, y, z):
    return x + 2 * y + 3 * z

print(partial_positionals(f, 4, SKIP, 6)(5))

outputs:

How to specify arg position for functool partial()

3 Answers3