understanding '*' "keyword only" argument notation in python3 functions

Question

I am having some difficulty behaviour of keyword only arguments feature in python3 when used with partial. Other info on keyword only arguments.

Here is my code:

def awesome_function(a = 0, b = 0, *, prefix):
    print('a ->', a)
    print('b ->', b)
    print('prefix ->', prefix)
    return prefix + str(a+b)

Here is my understanding of partial:

>>> two_pow = partial(pow, 2)
>>> two_pow(5)
32
>>>

What I understood is in the above example, partial makes the second argument to pow function as the only argument of two_pow.

My question is why does the following work:

>>> g = partial(awesome_function, prefix='$')
>>> g(3, 5)
a -> 3
b -> 5
prefix -> $
'$8'
>>>

But I get error in this:

>>> awesome_function(prefix='$', 3, 5)
  File "<stdin>", line 1
SyntaxError: non-keyword arg after keyword arg
>>>

I know that I can call awesome_function directly by

>>> awesome_function(prefix='$', a = 3, b = 5)
a -> 3
b -> 5
prefix -> $
'$8'
>>>

Answer 1

As per the semantics of the function calls in Python, the rules for the arguments to be passed are as follows

argument_list   ::=  positional_arguments ["," keyword_arguments]
                       ["," "*" expression] ["," keyword_arguments]
                       ["," "**" expression]
                     | keyword_arguments ["," "*" expression]
                       ["," keyword_arguments] ["," "**"     expression]
                     | "*" expression ["," keyword_arguments] ["," "**" expression]
                     | "**" expression

As you see here, the positional arguments should always appear at the beginning of the function calls. They cannot appear anywhere else. When you do

awesome_function(prefix='$', 3, 5)

it violates the above rule, as you are passing two positional arguments (3 and 5) after a keyword argument (prefix). That is why you are getting a SyntaxError, as Python is not able to parse the function call expression.

---

But, when you are using partial it works, because partial creates a new function object and it stores all the parameters to be passed to it. When you actually invoke the function object returned by partial, it applies all the positional arguments first and then the keyword arguments.

Answer 2

The error you get - SyntaxError: non-keyword arg after keyword arg - is because you tried to send positional arguments (like 3,5) after a keyword argument, that is not a valid syntax, and hence a syntax error. In the function call -

 awesome_function(prefix='$', 3, 5)
                          ^   ^  ^
                          |     These two are the positional argument.
                          ----- This is the keyword argument.                                

It works when using functools.partial because functools.partial knows to put positional arguments before keyword arguments , hence when you call the partial function - g() - with positional arguments , it sends those positional arguments before the keyword argument . Hence, in your case g(3, 5) ==> awesome_function(3, 5, prefix='$') .

A simple example to show this -

>>> from functools import partial
>>> def func(a=0,b=1):
...     print(a,b)
...
>>> ptfunc = partial(func,a=10)
>>> ptfunc(20)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: func() got multiple values for argument 'a'

In the above case, when we called ptfunc(20), 20 was passed as positional argument first, and then the keyword argument a=10 was passed, hence it complained that it got multiple values for the argument a.

---

And also as given in the documentation -

functools.partial is Roughly equivalent to:

def partial(func, *args, **keywords):
    def newfunc(*fargs, **fkeywords):
        newkeywords = keywords.copy()
        newkeywords.update(fkeywords)
        return func(*(args + fargs), **newkeywords)
    newfunc.func = func
    newfunc.args = args
    newfunc.keywords = keywords
    return newfunc