4

I am wrapping a C function which performs a blocking operation (select) and then handles incoming messages. My understanding is that when a C function is going to block, the correct way to call it while allowing other threads to run is:

Py_BEGIN_ALLOW_THREADS                                                  
blocking_function();
Py_END_ALLOW_THREADS

However, it happens that this function takes as a parameter a callback pointer. This callback is called on handling the incoming message that is pre-processed by the C function. I have successfully wrapped this callback in a function which calls PyEval_CallObject(), allowing me to pass it a Python callback.

Now that I'm adding threading support, I'm wondering if it's possible to simultaneously:

  • Release the GIL before calling this blocking operation.
  • Have this blocking operation safely call back into the python interpreter.

Is this going to cause problems? If so, is there a way around it?

Thanks.

Steve
  • 8,153
  • 9
  • 44
  • 91
  • It seems like the right approach might be to re-capture the GIL in my handler that calls `PyEval_CallObject`. However, this means passing forward the saved `PyThreadState`, which seems a bit tricky! – Steve Mar 01 '11 at 19:11

2 Answers2

7

I used these API functions several months ago, and my recollection is a bit hazy, but I believe this code will solve your problem. I am assuming version 2.x (3.x may be different):

PyGILState_STATE gstate;
gstate = PyGILState_Ensure();

/* Make your call to PyEval_CallObject() here (and any other PY API calls). */

PyGILState_Release(gstate);

(The above was taken from: Python C/API docs)

This is basically the inverse of the Py_BEGIN_ALLOW_THREADS/Py_END_ALLOW_THREADS macros. Within those, you release the GIL, but within the PyGILState functions you acquire the GIL.

dappawit
  • 12,182
  • 2
  • 32
  • 26
1

Works if I use a global to store the result of PyEval_SaveThread, and use this in the callback to call PyEval_RestoreThread. I just need to figure out how to more cleanly pass this to the callback in my code through the callback's context pointer.

Steve
  • 8,153
  • 9
  • 44
  • 91
  • Now that the PyGILState APIs are available, it is a *much* better idea to use those (as described in dappawit's answer). – ncoghlan Mar 02 '11 at 02:52