Are there any reasons not to mix Multiprocessing and Threading module in Python

Question

I am considering using Python to implement a program, which requires extensive multi-threading. Another requirement is that it will run on desktops, so having many processes, will make the application appear to be messy and harder to kill (in Task Manager). Therefore, I am considering using both the Threading and the Multiprocessing modules to reduce the number of processes. As far as I understand the GIL will apply only to a single process. My question is: Is there any reasons not to mix using the two modules?

ricopan · Accepted Answer · 2012-10-23T04:35:25.853

4

Note: This warning does not apply to windows.

Be careful! There is a nasty bug lurking with locks when combining threading and multiprocessing which is exposed when using the logging module as well. I've been bitten for the last week with child processes occasionally hanging. Now that I've disabled logging, so far so good (though not the best solution!).:

https://twiki.cern.ch/twiki/bin/view/Main/PythonLoggingThreadingMultiprocessingIntermixedStudy

http://bugs.python.org/issue6721

edited Oct 23 '12 at 04:35

answered Oct 19 '12 at 17:45

ricopan

662
6
13

6721 bug has nothing to do with threading/multiprocessing on Windows – jfs Oct 20 '12 at 01:30
Right, missed the windows tag. – ricopan Oct 23 '12 at 04:29
You just saved me hours of debugging... Thanks! – jbaiter Jul 05 '13 at 20:33

score 1 · Answer 2 · edited Jun 05 '11 at 19:13

1

Other than basic principle of KISS.... go for it, there shouldn't be any issues.

edited Jun 05 '11 at 19:13

David

4,786
11
52
80

answered Jun 05 '11 at 18:19

Ivan Novick

745
2
8
12

Are there any reasons not to mix Multiprocessing and Threading module in Python

2 Answers2

Linked