Embedding Python with C

Question

I want to use an event based Python library in a C application. I use the offical C-API for embedding Python: http://docs.python.org/2/c-api/index.html#c-api-index

It is no problem to call methods from C and collect return values. However, I don't know how to do the following:

Several of the python library functions take what I think is a python function pointer as an argument.

Is it possible, when calling these methods from C, to pass a C function pointer so the Python function uses a C function as callback?

If not, how to accomplish having Python use a C callback?

See http://stackoverflow.com/questions/6626167/build-a-pyobject-from-a-c-function (needs to be merged; out of mod points). — ecatmur, Nov 23 '12 at 23:36
@ecatmur The linked answer is incorrect. `methd` must be statically allocated because `PyCFunction` retains a pointer to the provided `PyMethodDef` and uses it later to retrieve the C function and flags. The code in the answer auto-allocates the `PyMethodDef`, which will cause a crash when the resulting Python function is invoked. — user4815162342, Nov 24 '12 at 08:27
@user4815162342: In the linked answer I think Manux was just showing an overview of the process. It's a bit silly, IMO, that Manux uses the function name as the module name instead of just `NULL`. In actual code I assume the `PyMethodDef` was allocated on the heap. — Eryk Sun, Nov 24 '12 at 20:59
@eryksun Maybe the answer should be clarified to note that `PyMethodDef` should be allocated on the heap. Which raises the followup question of when and how the dynamically allocated `PyMethodDef` will be released. My answer, with a bit of initial investment, resolves this issue nicely. — user4815162342, Nov 24 '12 at 22:51
Hi I have provided a nice example under this post https://stackoverflow.com/a/46441794/5842403 — Joniale, Dec 08 '17 at 08:10

user4815162342 · Accepted Answer · 2012-11-24T19:57:34.523

This is harder than one would expect, but it can be done.

If you have a single C function that you want to provide as a callback, you can use PyCFunction_New to convert it into a Python callable:

#include <python.h>

static PyObject *my_callback(PyObject *ignore, PyObject *args)
{
  /* ... */
}

static struct PyMethodDef callback_descr = {
  "function_name",
  (PyCFunction) my_callback,
  METH_VARARGS,                 /* or METH_O, METH_NOARGS, etc. */
  NULL
};

static PyObject *py_callback;

...
py_callback = PyCFunction_New(&callback_descr, NULL);

This approach won't work if you want to choose different callbacks at run-time, e.g. to provide a generic c_to_python function that converts a C callback function to a Python callback. In that case, you'll need to implement an extension type with its own tp_call.

typedef struct {
  PyObject_HEAD
  static PyObject (*callback)(PyObject *, PyObject *);
} CallbackObject;

static PyObject *
callback_call(CallbackObject *self, PyObject *args, PyObject *kwds)
{
  return self->callback(args, kwds);
}

static PyTypeObject CallbackType = {
    PyObject_HEAD_INIT(NULL)
    0,                          /*ob_size*/
    "Callback",                 /*tp_name*/
    sizeof(CallbackObject),     /*tp_basicsize*/
    0,                          /*tp_itemsize*/
    0,                          /*tp_dealloc*/
    0,                          /*tp_print*/
    0,                          /*tp_getattr*/
    0,                          /*tp_setattr*/
    0,                          /*tp_compare*/
    0,                          /*tp_repr*/
    0,                          /*tp_as_number*/
    0,                          /*tp_as_sequence*/
    0,                          /*tp_as_mapping*/
    0,                          /*tp_hash */
    (ternaryfunc) callback_call, /*tp_call*/
    0,                          /*tp_str*/
    0,                          /*tp_getattro*/
    0,                          /*tp_setattro*/
    0,                          /*tp_as_buffer*/
    Py_TPFLAGS_DEFAULT,         /*tp_flags*/
};

PyObject *
c_to_python(PyObject (*callback)(PyObject *, PyObject *))
{
  CallbackObject *pycallback = PyObject_New(CallbackObject, &CallbackType);
  if (pycallback)
    pycallback->callback = callback;
  return pycallback;
}

This code is trivially extended for the to also accept a user_data pointer; just store the user data in the CallbackObject struct.

Having a quick look, it seems boost::python does like this to wrap C functions. — neodelphi, Jul 21 '14 at 17:57
@neodelphi It makes sense that it does, but the question was tagged C, so `boost::python` doesn't really apply. BTW is `boost::python` actively maintained? The documentation seemed rather old when I checked it. — user4815162342, Jul 21 '14 at 20:30
You are right. Just found this post because I'm trying to implement a boost::python like library because it has not been updated since many years. I didn't find any other solution and as I see boost::python does this way, this seems to be the way to do. — neodelphi, Jul 25 '14 at 20:47

Embedding Python with C

1 Answers1