0

On Windows platform these two lines cause Python code execution to run in infinite loop with every loop starting from the beginning of the script:

import multiprocessing as mp
manager=mp.Manager()

It runs fine on Mac. What would be a reason? It successfully imports import multiprocessing as mp. But on manager=mp.Manager() it breaks (with no errors) and jumps back to the first line of the code.

alphanumeric
  • 17,967
  • 64
  • 244
  • 392

1 Answers1

2

On Windows, you have to protect calls to multiprocessing with if __name__ == "__main__":. See here for details on why this is required. This should work fine:

import multiprocessing as mp

if __name__ == "__main__":
    manager=mp.Manager()

Edit:

Note that putting your multiprocessing code inside functions you call from the if block is ok, too. Like this:

import multiprocessing as mp

def func(x):
   print("Got %s" % (x,))

def main():
  p = multiprocessing.Pool()
  results = p.map(func, range(0,5))

if __name__ == "__main__":
  manager = mp.Manager()
  main()
dano
  • 91,354
  • 19
  • 222
  • 219
  • So I have to move import into `if name==main` scope? – alphanumeric Jun 25 '14 at 16:50
  • @Sputnix No problem. Just note that **all** your calls that involve creating a new process via `multiprocessing` (like creating a `Pool` or `Process`) need to be done inside the `if __name__ ...` block. – dano Jun 25 '14 at 16:54
  • Should I put `pool.map_async(functName, ['arg1','arg2','arg3'])` inside of `if __name__=="__main__"` too? – alphanumeric Jun 25 '14 at 19:13
  • @Sputnix Yep, that too. – dano Jun 25 '14 at 19:14
  • And where do I declare `functName()` function that is called by `pool.map_async(functName, argList)`..... inside of `if __name__=="__main__"? Isn't it a mess? – alphanumeric Jun 25 '14 at 19:15
  • @Sputnix No, `functName()` can be defined outside of the `if` guard. Also keep in mind it's ok to call functions inside your `if` block to organize the everything better, if you want. I edited my answer to illustrate that. – dano Jun 25 '14 at 19:15
  • I am using multiprocessing dictionary variable declared with: `import multiprocessing as mp; manager=mp.Manager(); poolDict=manager.dict()` Can `poolDict` variable accessed from outside of `if __name__=='__main__'`? – alphanumeric Jun 25 '14 at 19:20
  • @Sputnix You should explicitly pass `poolDict` to the functions you're calling via `multiprocessing`. You can do this with `map`/`map_async` by using `functools.partial`. See [here](http://stackoverflow.com/a/24293334/2073595) for an example of this. – dano Jun 25 '14 at 19:24