python multiprocessing pool.map hangs

Question

I cannot make even the simplest of examples of parallel processing using the multiprocessing package run in python 2.7 (using spyder as a UI on windows) and I need help figuring out the issue. I have run conda update so all of the packages should be up to date and compatible.

Even the first example in the multiprocessing package documentation (given below) wont work, it generates 4 new processes but the console just hangs. I have tried everything I can find over the last 3 days but none of the code that runs without hanging will allocate more than 25% of my computing power to this task (I have a 4 core computer).

I have given up on running the procedure I have designed and need parallel processing for at this point and I am only trying to get proof of concept so I can build from there. Can someone explain and point me in the right direction? Thanks

Example 1 from https://docs.python.org/2/library/multiprocessing.html

#

from multiprocessing import Pool

def f(x):
    return x*x

if __name__ == '__main__':
    p = Pool()
    print(p.map(f, [1, 2, 3]))

Example 2 (modified from original) from http://chriskiehl.com/article/parallelism-in-one-line/

from multiprocessing import Pool

def fn(i):
    return [i,i*i,i*i*i]

test = range(10)

if __name__ == '__main__':    
    pool = Pool() 
    results = map(fn,test)
    pool.close() 
    pool.join()

I apologize if there is indeed an answer to this as it seems as though I should be able to manage such a modest task but I am not a programmer and the resources I have found have been less than helpful given my very limited level of knowledge. Please let me know what further information is needed.

Thank you.

The code looks perfectly fine. Can you try it using IDLE? Just to make sure it is not a spyder bug. — Maximilian Peters, May 30 '16 at 15:13
ok, the code from my second example seems to run just fine using IDLE. If this suggests a spyder bug how you would suggest proceeding as far as a UI to interface with python? — Kraig, May 30 '16 at 15:30

score 4 · Accepted Answer · edited May 23 '17 at 12:17

4

After installing spyder on my virtualmachine, it seems to be a spyder specific bug. Example 1 works in IDLE, executed via the command line, executed from within spyder (first saved and then executed), but not when executed line by line in spyder.

I would suggest simply to create a new file in spyder, add the lines of code, save it, and then run it..

For related reports see:

edited May 23 '17 at 12:17

Community

1
1

answered May 30 '16 at 15:57

Maximilian Peters

30,348
12
86
99

Big thanks for the guidance and the references, I appreciate it. I hadn't been considering spyder to be a potential source of the problem. – Kraig May 30 '16 at 16:34

python multiprocessing pool.map hangs

#

1 Answers1