5

Note that I'm constrained to use Python 2.6. I have a Python 2.6 application that uses a C++ multi-threaded API library built with boost-python. My use-case was simply to execute a Python function callback from a C++ boost thread but despite the many different attempts and researching all the available online resources I haven't found any way that works. All the proposed solutions revolve around a different combination of the functions: Py_Initialize*, PyEval_InitThreads, PyGILState_Ensure, PyGILState_Release but after trying all possible combinations nothing works in practice e.g.

Therefore, this question: how can I start and run a Python thread from C++? I basically want to: create it, run it with a Python target function object and forget about it.

Is that possible?

Community
  • 1
  • 1
SkyWalker
  • 13,729
  • 18
  • 91
  • 187

3 Answers3

1

Based on the text below from your question:

Run a Python thread from C++? I basically want to: create it, run it with a Python target function object and forget about it.

You may find useful to simply spawn a process using sytem:

system("python myscript.py")

And if you need to include arguments:

string args = "arg1 arg2 arg3 ... argn"
system("python myscript.py " + args)
otorrillas
  • 4,357
  • 1
  • 21
  • 34
0

You should call PyEval_InitThreads from your init routine (and leave the GIL held). Then, you can spawn a pthread (using boost), and in that thread, call PyGILState_Ensure, then call your python callback (PyObject_CallFunction?), release any returned value or deal with any error (PyErr_Print?), release the GIL with PyGILState_Release, and let the thread die.

void *my_thread(void *arg)
{
    PyGILState_STATE gstate;
    PyObject *result;

    gstate = PyGILState_Ensure();

    if ((result = PyObject_CallFunction(func, NULL))) {
        Py_DECREF(result);
    }
    else {
        PyErr_Print();
    }

    PyGILState_Release(gstate);

    return NULL;
}
pat
  • 12,587
  • 1
  • 23
  • 52
  • Thank you, but this doesn't answer this OP, it is another possible combination of what I have done already many many times and in many different ways but OK I will post a question update with what I get. Your answer has a couple of new spices to the Python-C++-concurrent-sup – SkyWalker Dec 20 '16 at 16:21
  • 1
    So you haven't tried *all* possible solutions then? – pat Dec 20 '16 at 16:29
  • Ah I know what the different spice is .. you are basically asking me to drop boost python and use the Python C interface directly. You add more complexity to my already complex problem :D. I have a `boost::python::object func` that I can simply call like `func(params)` – SkyWalker Dec 20 '16 at 16:46
  • 1
    Fine, replace the call to `PyObject_CallFunction` with the boost "simplification" layer, but you still need to deal with the GIL. Either use the C API directly for that, or use the RAII approach shown [here](https://www.medin.name/blog/2012/02/12/embedding-python-inside-a-multithreaded-c-program/) – pat Dec 20 '16 at 17:24
  • 1
    It looks like boost will turn python exceptions into the C++ exception `error_already_set`, so you should definitely use the RAII approach to release the GIL. You will likely want to catch the exception unless you prefer your program to terminate... – pat Dec 20 '16 at 17:39
  • 1
    You also need to be aware that boost python object destructors call `Py_DECREF` which can only be done while holding the GIL. Pay close attention to the order in which c++ destroys the RAII GIL wrapper and the boost python object wrapper returned from the boost python function object. – pat Dec 20 '16 at 18:13
0

The answer to the OP How to call Python from a boost thread? also answers this OP or DP (Derived:)). It clearly demonstrates how to start a thread from C++ that callbacks to Python. I have tested it and works perfectly though needs adaptation for pre-C++11. It uses Boost Python, is indeed an all inclusive 5* answer and the example source code is here.

Community
  • 1
  • 1
SkyWalker
  • 13,729
  • 18
  • 91
  • 187