Multiprocessing in Python 3.8

Question

I'm trying to use multiprocessing in python for the first time. I wrote a basic prime searching program and I want to run simultaneously it on each core. The problem is: when the program does the multiprocessing, it not only does the 'primesearch' function but also the beginning of the code. My expected output would be a list of prime numbers between 0 and a limit, but it writes 16 times (I have 16 cores and 16 processes) "Enter a limit: "

Here is my code:

import time
import os
from multiprocessing import Process

# Defining lists
primes = []
processes = []
l = [0]


limit = int(input('Enter a limit: '))

def primesearch(lower,upper):
    global primes
    for num in range(lower, upper):
        if num > 1:
            for i in range(2, num):
                if (num % i) == 0:
                    break
            else:
                primes.append(num)


# Start the clock
starter = time.perf_counter()

#Dividing data
step = limit // os.cpu_count()

for x in range(os.cpu_count()):
    l.append(step * (x+1))

l[-1] = limit

#Multiprocessing
for init in range(os.cpu_count()):
    processes.append(Process(target=primesearch, args=[l[init], l[init + 1],] ))

for process in processes:
    process.start()

for process in processes:
    process.join()


#End clock
finish = time.perf_counter()


print(primes)
print(f'Finished in {round(finish-starter, 2)} second')

What could be the problem?

Does this answer your question? [windows-trying-python-multiprocessing](https://stackoverflow.com/questions/18204782/) — stovfl, Apr 30 '20 at 20:36

score 2 · Accepted Answer · answered Apr 30 '20 at 20:35

2

You are using Windows - If you read the Python documenation for multiprocessing, it will reveal to you that you should protect your main code using if __name__==“__main__”: This is because on Windows each process re-executes the complete main .py file.

This is used in pretty much every example in the documentation., and explained in the section at the end ‘Programming guidelines’.

See https://docs.python.org/3/library/multiprocessing.html

answered Apr 30 '20 at 20:35

DisappointedByUnaccountableMod

6,656
4
18
22

That should always be the first place to look for information :-) – DisappointedByUnaccountableMod May 01 '20 at 09:38
Do you also have to protect main even if you do not Pool in main, but in a module imported by main? – JonnyRobbie Mar 11 '22 at 22:27

score 0 · Answer 2 · answered Apr 30 '20 at 23:01

Except for the __main__ issue, your way of using primes as a global list doesn't seem to work. I imported Queue from multiprocessing and used primes = Queue() and

    size = primes.qsize()
    print([primes.get() for _ in range(size)])
    primes.close()

in the main function and primes.put(num) in your function. I don't know if it's the best way, for me this works but if N > 12000 then the console freezes. Also, in this case, using multiprocessing is actually slightly slower than single process.

If you aim for speed, you can test only to the square root of num, which saves about half of the time. There are many optimizations you can do. If you are testing huge numbers, you can use the Rabin-Miller algorithm. http://inventwithpython.com/cracking/chapter22.html

Thank you for your answer. It is only a basic program and my goal was test the multiprocessing, not to search large primes. But anyway thank you for the link it seems very interesting! — Antal Nagy, May 01 '20 at 08:54

Multiprocessing in Python 3.8

2 Answers2