0

Using Python 3.4 on Linux

I’m new to parallel programming and I’m encountering problems when running the threading.Threads() for a specific method, and the module multiprocessing.Process() for another. Both methods work fine when the other one is commented out. Neither method has anything to do with the other (eg no attempt to share data). But when I have them both running neither works and everything freezes. As far as I can tell the multiprocessing seems to lock up. I assume the same thing applies for the Threading.

So the first step is to assert whether or not this is even possible?

(I have a feeling some of you will ask the reason for this... The threading does a simple capture user key checking while the multiprocessing does some heavy lifting)

I’m providing an example (more like pseudo code) to help illustrate how the methods are used.

file t.py

import threading

Class T:

    Def __init__():
        t = threading.Thread(target = self.ThreadMethod)
        t.daemon = True
        t.start()

    Def ThreadMehod():
        # capture key

file m.py

import multiproceessing

Class M:

    Def __init__():
        mp = multiprocessing.Process(target = self.ProcessMethod)
        mp.start()

    Def ProcessMethod():
        # heavy lifting

file main.py

import T
import M

Class main:

  Def __init__():
     T()

  Def DoTheProcess()
     for i in range(5):
       M()
Community
  • 1
  • 1
user1267259
  • 761
  • 2
  • 10
  • 22
  • 1
    the short answer is that i dont think this should happen, i've used both in the same application before without issue, although with a slightly different arrangement, have you checked to make sure you haven't got any overlap of variable names? and how are you importing? are you doing from x import * or just import x? – James Kent Jan 12 '15 at 23:47
  • I've just tried using Pool instead of Process and now it seems that both work... Like I said, I'm new to mp so I probably implemented Process wrong. I'd accept your answer if I could... – user1267259 Jan 13 '15 at 00:59
  • Could you provide [a minimal but complete code example](http://stackoverflow.com/help/mcve)? It might help to diagnose the problem. Mention what is your OS and how do you run the script: what do you expect to happen? And what happens instead? – jfs Jan 13 '15 at 17:57
  • Can you reproduce the issue with `forkserver` start method? (make sure to call `mp.set_start_method('spawn')` before starting any threads). – jfs Jan 13 '15 at 18:01
  • I appreciate the help J.F. Sebastian, but the purpose with this question was to rule out compatibility problems. I was going to ask a new question more specific for the code in question but as the pooling works I thought I'd just leave it be for the moment. However, if you're interested I could provide more code. – user1267259 Jan 13 '15 at 20:55
  • 1
    If `Pool` works but `Process` doesn't then it means that the issue is in your code. The answer to the title is "no. `threading` and `multiprocessing` are not mutually exlusive. Though there are known issues (e.g., the reason for `atfork` existance) that constrain how they can be used together." – jfs Jan 14 '15 at 08:22

1 Answers1

0

"no. threading and multiprocessing are not mutually exlusive. Though there are known issues (e.g., the reason for atfork existance) that constrain how they can be used together." - J.F. Sebastian

user1267259
  • 761
  • 2
  • 10
  • 22