How does PyArg_ParseTupleAndKeywords work?

Question

I've been trying to learn how to write C-extensions for Python and want to be sure I understand how PyArg_ParseTupleAndKeywords works.

I believe that the first argument is a PyObject pointer that points to an array of the arguments being passed into the C-extension function in the order they were passed. The second argument is a list of keywords that were passed, the positions at which they were passed and, very likely, some sort of indicator flag telling at which position the keywords begin and position becomes irrelevant.

PyArg_ParseTupleAndKeywords then uses its list of keywords (4th argument) to map between arguments specified with a keyword and both the format string (3rd argument) and addresses of C variables (5th & + arguments) to which the appropriate values should be copied.

Is my understanding correct? When I read through the online documentation, all I see are references to "positional arguments and keyword arguments", which leave me feeling a little bit in the dark. Where is the file for the Python interpreter that handles PyArg_ParseTupleAndKeywords?

score 21 · Answer 1 · edited Jun 14 '19 at 08:09

21

To emulate the following in python:

def keywords(a, b, foo=None, bar=None, baz=None):
    pass

The following will work:

static PyObject *keywords(PyObject *self, PyObject *args, PyObject *kwargs)
{
    char *a;
    char *b;
    char *foo = NULL;
    char *bar = NULL;
    char *baz = NULL;

    // Note how "a" and "b" are included in this
    // even though they aren't supposed to be in kwargs like in python
    static char *kwlist[] = {"a", "b", "foo", "bar", "baz", NULL};

    if (!PyArg_ParseTupleAndKeywords(args, kwargs, "ss|sss", kwlist, 
                                     &a, &b, &foo, &bar, &baz)) 
    {
        return NULL;
    }

    printf("a is %s\n", a);
    printf("b is %s\n", b);
    printf("foo is %s\n", foo);
    printf("bar is %s\n", bar);
    printf("baz is %s\n", baz);

    Py_RETURN_NONE;
}

// ...

static PyMethodDef SpamMethods[] = 
{
    // ...
    {"keywords", (PyCFunction) keywords, METH_VARARGS | METH_KEYWORDS, "practice kwargs"},
    {NULL, NULL, 0, NULL}
    // ...
}

And to use it:

from spam import keywords

keywords()         # Fails, require a and b
keywords('a')      # fails, requires b
keywords('a', 'b')
keywords('a', 'b', foo='foo', bar='bar', baz='baz') 
keywords('a', 'b','foo', 'bar', 'baz')
keywords(a='a', b='b', foo='foo', bar='bar', baz='baz')

edited Jun 14 '19 at 08:09

3dGrabber

4,710
1
34
42

answered Feb 12 '17 at 21:19

Matthew Moisen

16,701
27
128
231

Weird— I can't edit. When you cast the "keywords" function, you want to cast it as `PyCFunctionWithKeywords`, rather than `PyCFunction`. – Jim Pivarski Jun 16 '17 at 14:04
1

@JimPivarski I seems like it would have to be `PyCFunction` https://stackoverflow.com/questions/10264080/python-c-extension-why-are-methods-that-use-keyword-arguments-cast-to-pycfunct – railgun Sep 18 '18 at 03:53
4

Setting `language = "c++"` on `setup.py` and build it with `-std=c++11` throws this warning `warning: ISO C++ forbids converting a string constant to ‘char*’ [-Wwrite-strings]` for the line `static char *kwlist[] = {"", "", NULL};` – Evandro Coan Jun 10 '19 at 18:45
C++ has some different rules around const. In this case, use `const char *kwlist[]` instead of `static char *kwlist`. – the paul Jan 10 '22 at 07:04
@user, try `const_cast(kwlist),` – Alexander Lubyagin Feb 03 '23 at 15:29

score 8 · Accepted Answer · answered May 16 '12 at 20:12

Have you read through the opening explanation at http://docs.python.org/c-api/arg.html ? It does a pretty good job of explaining what's going on. Don't go right to the specific reference for PyArg_ParseTupleAndKeywords; it assumes you read the text above, and isn't very helpful by itself.

You've almost got it, though. The first argument is indeed a list of incoming positional arguments. The second is a map of incoming keyword arguments (mapping given keyword name to given value). The fourth argument is actually the list of keywords which your function is prepared to accept. Yes, the 3rd argument is the format string and the 5th and later are C pointers into which the values are copied.

You'll find PyArg_ParseTupleAndKeywords() under Python/getargs.c.

Rather than just the C API doc, there is a good readable summary at http://docs.python.org/2/extending/extending.html — timbo, Jun 06 '13 at 01:35
The documentation at http://docs.python.org/c-api/arg.html is pretty bad. — smbear, May 08 '20 at 10:19

How does PyArg_ParseTupleAndKeywords work?

2 Answers2

Linked