1

I'm trying to interface to a legacy multi-threaded Python application from some new C++ code being developed. I cannot seem to get C++ to Python calls to work reliably if the Python code has started a thread. In pseudocode, what I am trying to get working is:

  • C++ binary starts running
  • C++ binary initializes Python environment and imports the module(s) it needs
  • Python starts one or more threads
  • At some stage later, C++ makes one or more calls to Python functions
  • ...
  • C++ program cleans up Python environment & exits.

The code I have so far successfully calls a Python function multiple times from C++ successfully, but crashes on exit, and also the single thread created in Python seems to be stalled until the right before the C++ program exits.

The current calls I'm making to setup the Python environment are:

  // Initialize Python environment
  Py_Initialize();

  // Initialize threading and acquire Python GIL
  PyEval_InitThreads();

  PyThreadState * mainThreadState = NULL;
  // save a pointer to the main PyThreadState object
  mainThreadState = PyThreadState_Get();
  // release the lock
  PyEval_ReleaseLock();

  // get the global lock
  PyEval_AcquireLock();
  // get a reference to the PyInterpreterState
  PyInterpreterState* mainInterpreterState = mainThreadState->interp;
  // create a thread state object for this thread
  PyThreadState* myThreadState = PyThreadState_New(mainInterpreterState);
  // free the lock
  PyEval_ReleaseLock();

  PySys_SetArgv(argc, argv);

A Python function is then called a number of times like this:

  // grab the global interpreter lock
  PyEval_AcquireLock();
  // swap in my thread state
  PyThreadState_Swap(myThreadState);

  PyObject* pMod = PyImport_ImportModule(moduleName.c_str());
  PyObject* pfn = PyObject_GetAttrString(pMod, fnName.c_str());

  // Create the data structure to pass the single C string
  PyObject* pargs = Py_BuildValue("(s)", arg.c_str());
  PyObject* result = PyEval_CallObject(pfn, pargs);

  Py_DECREF(pargs);
  Py_DECREF(pfn);
  Py_DECREF(pMod);

  // clear the thread state
  PyThreadState_Swap(NULL);
  // release our hold on the global interpreter
  PyEval_ReleaseLock();

The one-time cleanup code called when the C++ application is about to exit is:

  PyEval_AcquireLock();
  Py_Finalize();

It would be great if someone could point me to some sample code that does what I am trying to achieve, or could point out what I am doing wrong in the above steps?

Cœur
  • 37,241
  • 25
  • 195
  • 267
fred basset
  • 9,774
  • 28
  • 88
  • 138
  • I ended up getting it working by using the techniques from the first answer to this other SO post :http://stackoverflow.com/questions/29595222/multithreading-with-python-and-c-api. Code example : https://gist.github.com/sterin/e8090d0451ab781a4e22 – fred basset Apr 21 '17 at 23:59

0 Answers0