Multiprocess code in python fails to execute on windows side

Question

I have written a very basic code to test multiprocess in python. When i try to run the code on my windows machine, it does not run while it works fine on linux machine. Below is the code and the error that it throws.

from multiprocessing import Process
import os
import time

# creating a list to store all the processes that will be created
processes = []

# Get count of CPU Cores
cores = os.cpu_count()
def square(n): #just creating a random program for demostration
    for i in range (n):
        print(i)
        i*i
        time.sleep(0.1)

# create a process.
for i in range(cores):
    p = Process(target=square,args=(100,)) 
    processes.append(p)

#statrting all the processes
for proc in processes:
    proc.start()

# join process
for proc in processes:
    proc.join()

print("All processes are done")

Due to inability of doing a fork, multiprocessing works differently in Windows and your module is executed again. You have protect the parts that create the processes with `if __name__ == '__main__':` or you will fall into a recursion. — Klaus D., Aug 12 '22 at 08:25
related: https://stackoverflow.com/questions/24374288/where-to-put-freeze-support-in-a-python-script — Paul Rooney, Aug 12 '22 at 08:27
Read the `multiprocessing` documentation carefully. Especially the sections "Contexts and start methods" and "Programming guidelines". — Roland Smith, Aug 12 '22 at 08:33

Cow · Accepted Answer · 2022-08-12T08:38:59.937

Most Process executions (Not Threads) on Python will start a new instance importing itself. This means your global code will be executed every time the instance is doing the import. (This only applies to the spawn start method)

In order to avoid these issues, you have to move your code into the if __name__ == "__main__": function in order for the Process to create a new instance correctly.

You fix it like so:

from multiprocessing import Process
import os
import time

def square(n): #just creating a random program for demostration
    for i in range (n):
        print(i)
        i*i
        time.sleep(0.1)

if __name__ == "__main__":
    # Get count of CPU Cores
    cores = os.cpu_count()
    # creating a list to store all the processes that will be created
    processes = []
    # create a process.
    for i in range(cores):
        p = Process(target=square,args=(100,)) 
        processes.append(p)
    #statrting all the processes
    for proc in processes:
        proc.start()
    # join process
    for proc in processes:
        proc.join()
    print("All processes are done")

Result:

1
0
0
1
1
0
2
0
0
1
1
1
2
... Truncated
All processes are done

Your first paragraph only applies to the *spawn* start method. — Roland Smith, Aug 12 '22 at 08:34
Thank you so much for the detailed explanation. I however have doubt that why it works well without any changes in linux environment? — Satish Kumar Singh, Aug 12 '22 at 08:50
@SatishSingh Because multiprocessing on Windows is very different than Linux. You should read some of the documentation posted in the comments. — Cow, Aug 12 '22 at 08:51

Multiprocess code in python fails to execute on windows side

1 Answers1