Unexpected error in Python code with multiprocessing

Question

I get a error when I run this code.

from multiprocessing import Process, cpu_count
import time


def counter(num):
    count = 0
    while count < num:
        count += 1


def main():

   print("cpu count:", cpu_count())

    a = Process(target=counter, args=(500000000,))
    b = Process(target=counter, args=(500000000,))

    a.start()
    b.start()

   print("processing...")

    a.join()
    b.join()

    print("Done!")
    print("finished in:", time.perf_counter(), "seconds")


main()

I was expecting Python to print up to 1000000000 but it just gives me a unexpected error. I am not sure what if name == "main": does so I have not used it.

"I am not sure what `if __name__ == "__main__":` does so I have not used it." Removing that would be the first problem :) See also [Compulsory usage of `if __name__=="__main__"` in windows while using multiprocessing](https://stackoverflow.com/q/20360686/11082165) and linked questions. — Brian61354270, Jul 22 '23 at 00:34
_but it just gives me a unexpected error_ Are we supposed to guess what the error is? — John Gordon, Jul 22 '23 at 00:37
if __name__ == '__main__': freeze_support() ... The "freeze_support()" line can be omitted if the program is not going to be frozen to produce an executable. — Skrug Nuggsta, Jul 22 '23 at 00:42
You seem to have misunderstood how time.perf_counter() works — DarkKnight, Jul 22 '23 at 06:09

score 2 · Answer 1 · answered Jul 22 '23 at 00:45

2

Your problem is that you must write:

if __name__ == '__main__':
    main()

On non-Linux machines, Python starts new processes but having them re-read the main file. In your original code, each time a new process is started, it starts up another ten processes.

You need to make sure that main() is only executed once.

answered Jul 22 '23 at 00:45

Frank Yellin

9,127
1
12
22

1

_"On non-Linux machines, Python starts new processes"_ fun fact, that fact this this _isn't_ the default on unix systems is [now considered a mistake](https://github.com/python/cpython/issues/84559). Current sites are on making `spawn` the default start method on unix systems as well in Python 3.14. – Brian61354270 Jul 22 '23 at 00:54
@Brian61354270. Good to know. – Frank Yellin Jul 22 '23 at 04:05

score 1 · Answer 2 · answered Jul 22 '23 at 06:20

If you want to measure a duration you need to note both start and end times then subtract one from the other.

Something like this:

from multiprocessing import Pool
from time import perf_counter

def counter(num):
    for _ in range(num):
        ...
    return num

N = 500000000
NPROCS = 2

def main():
    start = perf_counter()

    with Pool() as pool:
        for ar in [pool.apply_async(counter, (N,)) for _ in range(NPROCS)]:
            print(ar.get())

    end = perf_counter()

    print(f'Duration {end-start:.2f}s')

if __name__ == '__main__':
    main()

Output:

500000000
500000000
Duration 8.61s

Unexpected error in Python code with multiprocessing

2 Answers2