I'm trying to interface to a legacy multi-threaded Python application from some new C++ code being developed. I cannot seem to get C++ to Python calls to work reliably if the Python code has started a thread. In pseudocode, what I am trying to get working is:
- C++ binary starts running
- C++ binary initializes Python environment and imports the module(s) it needs
- Python starts one or more threads
- At some stage later, C++ makes one or more calls to Python functions
- ...
- C++ program cleans up Python environment & exits.
The code I have so far successfully calls a Python function multiple times from C++ successfully, but crashes on exit, and also the single thread created in Python seems to be stalled until the right before the C++ program exits.
The current calls I'm making to setup the Python environment are:
// Initialize Python environment
Py_Initialize();
// Initialize threading and acquire Python GIL
PyEval_InitThreads();
PyThreadState * mainThreadState = NULL;
// save a pointer to the main PyThreadState object
mainThreadState = PyThreadState_Get();
// release the lock
PyEval_ReleaseLock();
// get the global lock
PyEval_AcquireLock();
// get a reference to the PyInterpreterState
PyInterpreterState* mainInterpreterState = mainThreadState->interp;
// create a thread state object for this thread
PyThreadState* myThreadState = PyThreadState_New(mainInterpreterState);
// free the lock
PyEval_ReleaseLock();
PySys_SetArgv(argc, argv);
A Python function is then called a number of times like this:
// grab the global interpreter lock
PyEval_AcquireLock();
// swap in my thread state
PyThreadState_Swap(myThreadState);
PyObject* pMod = PyImport_ImportModule(moduleName.c_str());
PyObject* pfn = PyObject_GetAttrString(pMod, fnName.c_str());
// Create the data structure to pass the single C string
PyObject* pargs = Py_BuildValue("(s)", arg.c_str());
PyObject* result = PyEval_CallObject(pfn, pargs);
Py_DECREF(pargs);
Py_DECREF(pfn);
Py_DECREF(pMod);
// clear the thread state
PyThreadState_Swap(NULL);
// release our hold on the global interpreter
PyEval_ReleaseLock();
The one-time cleanup code called when the C++ application is about to exit is:
PyEval_AcquireLock();
Py_Finalize();
It would be great if someone could point me to some sample code that does what I am trying to achieve, or could point out what I am doing wrong in the above steps?