1

I have a project for my college regarding multiprocessing in python.For my python projects I use spyder in windows. Therefore im trying to run a very trivial multi - processing code in spyder but every time i run it spyder console freezes and never finishes. This is my code:

from multiprocessing import Pool, freeze_support
import multiprocessing

def f(i):
    return i+1

def main():
    pool = multiprocessing.Pool(4)
    result = pool.map(f, range(4))
    pool.close()
    pool.join()
    print result

if __name__ == '__main__':
    freeze_support() #you need this in windows
    main()

I have noticed this is a common issue with multi-proccesing and the lack of fork in windows, therefore i took into account the paragraph 16.6.3.2. Windows in https://docs.python.org/2/library/multiprocessing.html#windows .

I also avoided printing from my child process as stated here:Simple Python Multiprocessing function doesn't output results and instead used return.

My code works if i run it from a unix terminal, but i'm mainly using windows for my python projects. Is there a workaround for this issue in windows?

vtsimp
  • 132
  • 2
  • 16
  • 1
    Have you tried running from Windows Command Prompt? Or using Python version 3? I tried running your code from Windows 10 Command Prompt, Python version 3 and it works. – hamster ham Oct 11 '18 at 15:35
  • 1
    I have and it works fine. Im trying to find a solution or a workaround for the spyder though. After some research i found out that it is a live intepreter issue. But i guess not using spyder for multi processing isnt exactly a programming answer. – vtsimp Oct 11 '18 at 16:26
  • 1
    for what it's worth, I can run your code just fine on win7, py2.7, spyder 3.1.3 in its ipython interpreter – kevinsa5 Oct 11 '18 at 20:55
  • 1
    @kevinsa5 i am on win10,python2.7,spyder 3.2.6 and the issue persists. [Here](https://stackoverflow.com/questions/48078722/no-multiprocessing-print-outputs-spyder?noredirect=1&lq=1) it is stated by a spyder maintainer that indeed multiprocessing doesn't work well on Windows in Spyder's IPython console – vtsimp Oct 11 '18 at 21:57
  • @vtsimp could you please open an Issue about your problem on Spyder's GitHub repo. I'll answer your question over there. – Jean-Sébastien Oct 12 '18 at 12:05

1 Answers1

2

After some research i understand that there is an issue with interactive interpreters and multiporcessing.

In: https://docs.python.org/2.7/library/multiprocessing.html#introduction it is stated that: Note: ... some examples, such as the Pool examples will not work in the interactive interpreter.

An older post addressing a similar issue it was answered by a spyder maintainer that indeed multiprocessing doesn't work well on Windows in Spyder's IPython console. (No multiprocessing print outputs (Spyder))

The only workaround i found so far is to use the windows cmd to run the code

vtsimp
  • 132
  • 2
  • 16