0

Similarly to the question discussed here, I need to call asynchronously python code from a multithreaded C++ program. But it will not be the main thread (the one that calls Py_Initialize() and PyEval_InitThreads()) that will call for the python execution.

I don't know how to manage the calls to PyEval_SaveThread() and PyEval_RestoreThread(): is it safe to call PyEval_SaveThread() and leave the retrieved thread state unused? Would a call to PyThreadState_Clear() make sense?

I would like to only manage the GIL using PyGILState_Ensure() and PyGILState_Release() if possible.

Thank you!

Edit 1:

I tried that in the main thread:

PyEval_InitThreads();
Py_Initialize();

PyThreadState* _state = PyEval_SaveThread();
PyEval_AcquireLock();
PyThreadState_Clear(_state);
PyEval_ReleaseLock();

I get a segmentation fault.

Edit 2:

I did not find a way to abandon the PyThreadState just after the PyEval_SaveThread(), but I confirm that it is possible to only deal with PyGILState_Ensure() and PyGILState_Release() to protect a python execution once the thread is saved, no matter if we are still in the same thread or not.

At the end, it seems mandatory to restore the thread before finalization.

Edit 3:

  • Today, using Python 2.7, moving to Python 3 in a near future.
  • regarding the segmentation fault while clearing the python thread state, there is no error message. The seg fault appears at finalization. Python may guess that it has to close the main thread that I just cleared?
  • When the program terminate, if no error occurred, the python executions are over, and then it should be able to finalize nicely. It is the way it runs, when I do not clear the thread state.
Community
  • 1
  • 1
Grumot
  • 81
  • 1
  • 7
  • A few questions: 1. What version of Python is this intended for? 2. Is there an error message? 3. What happens when your program terminates? e.g. would you like to terminate python. – sterin Feb 08 '17 at 09:53
  • @user99279, thank you, I added the informations in the question... – Grumot Feb 09 '17 at 11:08

1 Answers1

0

I'm also learning, so the information here may not be fully complete, but at least I could overcome the segmentation fault.

Using Python 2.7 C API.

In the main():

Py_Initialize();
PyEval_InitThreads();

// IMPORTNT: You must release before launching threads
PyEval_ReleaseLock();

// Create/Start threads here

// Wait until threads ends

// Before finishing, acquire the GIL
PyGILState_STATE gstate = PyGILState_Ensure();
Py_Finalize();

In your thread's run method:

PyGILState_STATE gstate = PyGILState_Ensure();

// Your Python work, i.e:
PyRun_SimpleString("print('Hello from Thread')");

PyGILState_Release(gstate);

Note: Currently I have some troubles when the Python code that I'm calling from my threads involves importing modules and creating pyplot objects, but that is another story.

Juampa
  • 1
  • 2
  • Thank you, nice workaround directly releasing the lock first... I will test it very soon; it sounds promising – Grumot Jul 13 '17 at 09:23