7

I'm running some code on both a Windows 7 pc and a Windows 10 laptop:

def project(filename):
    **Do Something Here**


if __name__ == '__main__':
    pool = Pool(processes=4)
    results = [pool.apply_async(project, args=(filename,)) for filename in filenamelist]
    output = [p.get() for p in results]
    print output

Both computers are dual core/4 threads, so they should both be fine running 4 processes. The problem I have is that when I run the code on Windows 10 I get 4 python processes running, but they use 0% of the cpu and they will not output anything, unlike the Windows 7 pc which will run at full usage on all 4 threads and work perfectly.

The code works fine on the Windows 10 laptop if I don't use multiprocessing, so the problem must be related to that. Does multiprocessing with Python not work in Windows 10 yet? I am running Python 2.7 on both machines by the way.

[Edit]: Windows 7 pc processor is an i5-650, Windows 10 laptop processor is an i3-2370M

[Update]: I reverted the laptop back to Windows 8.1 and the exact same code runs as intended, this is definitely a Windows 10 issue.

[Edit]: The method I'm using to generate the filenamelist is as follows, however this works fine on Windows 7.

def get_unfinished_files(indir, outdir):
    filenamelist = []
    for filename in os.listdir(indir):
        if filename not in os.listdir(outdir) and filename.endswith('.png'):
            filenamelist.append(filename)
    return filenamelist
adam b
  • 346
  • 1
  • 7
  • 17
  • The multiprocessing code you've shown works for me on Windows 10 with Python 2.7.10. I made `project` simply `print` the filename it is passed and it worked with no problems. Perhaps the issue is somewhere else (either in the contents of `project` or perhaps in how you're generating `filenamelist` in the main code)? – Blckknght Sep 04 '15 at 00:22
  • It doesn't seem to be anything in `project` that is the issue, as I can run the code without using multiprocessing and it works fine then. You might be on the right track with `filenamelist` though, since that's the only other difference between running with multiprocessing and running normally. I'll edit my post to show the method for generating `filenamelist`. – adam b Sep 04 '15 at 00:31
  • 2
    Can you provide a full example that can be run by anyone with Windows 10? – Asad Saeeduddin Sep 04 '15 at 00:40
  • Does `project` get executed in windows 10 + multiprocessing? add some logging.... Long shot: python.exe directory name has some spaces / strange characters on windows 10 which generates some issues for running it correctly. – Udi Oct 26 '15 at 12:19

1 Answers1

2

Had similar issue. As I found, it was just a bug in python in my case: https://bugs.python.org/issue35797

It occurs when using multiprocessing through venv.

Bugfix is released in Python 3.7.3.

Name
  • 124
  • 12