8

I'm trying to test a multi-threaded C++ DLL. This DLL is supposed to be thread-safe. I have it wrapped with boost.python, and I'd like to create multiple python threads to exercise the DLL through the boost.python wrapper. I'm actually trying to cause threading problems.

What I can't seem to find good documentation on is whether the python interpreter will support two of its threads (on different cores, say) calling into an imported module concurrently, and whether the GIL needs tending at all given that I don't want any added safety above what the DLL is supposed to provide.

Can anyone describe or refer me to a description of python calling DLL modules from multiple threads and how the GIL is suppsed to be used in this case?

jspencer
  • 522
  • 4
  • 14
  • Apparently you will have to release the GIL by yourself, otherwise you will have no more than one thread running at a time. See http://stackoverflow.com/questions/1576737/releasing-python-gil-in-c-code – lvella Dec 12 '11 at 20:38
  • 1
    Is this a duplicate of http://stackoverflow.com/questions/8009613/ ? – Matthew Scouten Dec 12 '11 at 22:11
  • A side question: if I don't release the GIL, and the C++ code called from Python creates a thread, can this thread call python code safely or not. I suppose not as it would explain a crash I got... – MatthieuW Dec 14 '11 at 10:42

2 Answers2

4

How to release GIL when calling a C++ function from Python via Boost.Pyhton:

http://wiki.python.org/moin/boost.python/HowTo#Multithreading_Support_for_my_function

lvella
  • 12,754
  • 11
  • 54
  • 106
1

The answer is no, the GIL will never truly multi-thread unless the DLL manually releases the lock. Python allows exactly one thread to run at a time unless the extension manually says, "I'm blocked, carry on without me." This is commonly done with the Py_BEGIN_ALLOW_THREADS macro (and undone with Py_END_ALLOW_THREADS) defined in python's include/ceval.h. Once an extension does this, python will allow another thread to run, and the first thread doing any python stuff will likely cause problems (as the comment question notes.) It's really meant for blocking on I/O or going into heavy compute time.

jspencer
  • 522
  • 4
  • 14