Getting `BrokenProcessPool` error in a `concurrent.futures` example

Question

The example I am running is mentioned in this PyMOTW3 link. I am reproducing the code here:

from concurrent import futures
import os


def task(n):
     return (n, os.getpid())


ex = futures.ProcessPoolExecutor(max_workers=2)
results = ex.map(task, range(5, 0, -1))
for n, pid in results:
    print('ran task {} in process {}'.format(n, pid))

As per source, I am supposed to get following output:

ran task 5 in process 40854
ran task 4 in process 40854
ran task 3 in process 40854
ran task 2 in process 40854
ran task 1 in process 40854

Instead, I'm getting a long message with following concluding line -

concurrent.futures.process.BrokenProcessPool: A process in the process pool was terminated abruptly while the future was running or pending.

I am using Windows machine and running Python 9. All other examples are otherwise running fine. What is going wrong here?

score 0 · Answer 1 · answered Jan 06 '23 at 06:50

I've finally been able to resolve the issue. The issue seems to be Windows specific. Following a related Stack Overflow post, I used if __name__=="__main__" idiom. The modified code is:

from concurrent import futures
import os

def task(n):
    return (n, os.getpid())

def main():
    ex = futures.ProcessPoolExecutor(max_workers=2)
    results = ex.map(task, range(5, 0, -1))

    for n, pid in results:
        print('ran task {} in process {}'.format(n, pid))
    
if __name__ == '__main__':
    main()

It worked, although I'm still not sure why this worked.

Getting `BrokenProcessPool` error in a `concurrent.futures` example

1 Answers1