Why is this message printed more than once during multiprocessing with concurrent.futures.ProcessPoolExecuter()?

Question

The statement "I should appear only once" should appear only once. I am not able to understand why it appears 3 more times... It's clear to me that my code is executing 3 further processes. But in these 3 processes only funktion0() is getting called. Why does the statement "I should appear only once" get included in these extra 3 processes? Could someone explain?

Code:

from datetime import datetime
#print(datetime.now().time())

from time import time, sleep
#print(time())
print("I should appear only once")
from concurrent import futures


def funktion0(arg0):
    sleep(arg0)
    print(f"ich habe {arg0} sek. gewartet, aktuelle Zeit: {datetime.now().time()}")

if __name__=="__main__":

    with futures.ProcessPoolExecutor(max_workers=3) as obj0:
        obj0.submit(funktion0, 5)
        obj0.submit(funktion0, 10)
        obj0.submit(funktion0, 15)
        obj0.submit(funktion0, 20)
        print("alle Aufgaben gestartet")

    print("alle Aufgaben erledigt")

Expected output:

I should appear only once
alle Aufgaben gestartet
ich habe 5 sek. gewartet, aktuelle Zeit: 18:32:51.926288
ich habe 10 sek. gewartet, aktuelle Zeit: 18:32:56.923648
ich habe 15 sek. gewartet, aktuelle Zeit: 18:33:01.921168
ich habe 20 sek. gewartet, aktuelle Zeit: 18:33:11.929370
alle Aufgaben erledigt

Actual output:

I should appear only once
alle Aufgaben gestartet
I should appear only once
I should appear only once
I should appear only once
ich habe 5 sek. gewartet, aktuelle Zeit: 18:32:51.926288
ich habe 10 sek. gewartet, aktuelle Zeit: 18:32:56.923648
ich habe 15 sek. gewartet, aktuelle Zeit: 18:33:01.921168
ich habe 20 sek. gewartet, aktuelle Zeit: 18:33:11.929370
alle Aufgaben erledigt

please tag your question as windows, as it's probably is windows-related only. — Jean-François Fabre, Feb 02 '19 at 17:49

Jean-François Fabre · Accepted Answer · 2019-02-02T18:40:35.017

It's the classic issue with windows (RuntimeError on windows trying python multiprocessing) only less dramatic.

When you're using multiprocessing on windows, the process isn't forked but duplicated (because fork doesn't exist in windows OS) using some tricky mechanism to "emulate" fork but not accurately because OS doesn't allow it (What's the best way to duplicate fork() in windows?).

So the statement is printed as many times as there are processes, unless you protect it with __name__ == "__main__"

(you can probably speed up workers startup by moving most import statements in that scope too)

Why is this message printed more than once during multiprocessing with concurrent.futures.ProcessPoolExecuter()?

1 Answers1