how to use multiprocessing in python right?

Question

import time
from multiprocessing import Process
start = time.perf_counter()


def sleep():
    print('Sleeping 1 second(s)...')
    time.sleep(1)
    return 'Done Sleeping...'

p1 = Process(target = sleep)
p2 = Process(target = sleep)
p1.start()
p2.start()
p1.join()
p2.join()
finish = time.perf_counter()
print(f'Finished in {round(finish-start, 2)} second(s)')

output:

Finished in 0.17 second(s)

I tried to use multiprocessing, but when I run the code it`s over in 0.17~ seconds and not 1 as it supposed to be, it's not sets off the function at all...

If I will put brackets like this :

p1 = Process(target = sleep())
p2 = Process(target = sleep())

output:

Sleeping 1 second(s)...
Sleeping 1 second(s)...
Finished in 2.35 second(s)

windows 10. python 3.7.4 thank you:)

Check https://stackoverflow.com/questions/30075344/time-sleep-inaccurate-for-python-counter — graphite, Oct 16 '19 at 13:23
Your code is running as expected for me too, under python 3.8. ```Finished in 1.16 second(s)``` — Kevin Müller, Oct 16 '19 at 13:26
The reason your 2nd version is taking ~2 seconds is because `sleep()` is being evaluated in the main thread and the return value of `sleep()` is being passed as the target. — SyntaxVoid, Oct 16 '19 at 14:49

Artiom Kozyrev · Answer 1 · 2019-10-16T14:11:24.913

5

I have solved the problem, in order to make your code work you should add if __name__ == '__main__'. Both of your new processes need to get access to your def sleep() in order to do it you must either separate "executable" part of your code by __name__ == "__main__" or put def sleep() in another file and export it from there from filename import sleep

import time
from multiprocessing import Process
start = time.perf_counter()


def sleep():
    print('Sleeping 1 second(s)...')
    time.sleep(1)
    return 'Done Sleeping...'


if __name__ == "__main__":
    p1 = Process(target = sleep)
    p2 = Process(target = sleep)
    p1.start()
    p2.start()
    p1.join()
    p2.join()
    finish = time.perf_counter()
    print(f'Finished in {round(finish-start, 2)} second(s)')

Hope the answer is useful for you.

Site form book "The Python 3 Standard Libaray by Example" by Doug Hellmann:

One difference between the threading and multiprocessing examples is the extra protection for __main__ included in the multiprocessing examples. Due to the way the new processes are started, the child process needs to be able to import the script containing the target function. Wrapping the main part of the application in a check for __main__ ensures that it does not run recursively in each child as the module is imported. Another approach is to import the target function from a separate script. For example, multiprocessing_import_main.py uses a worker function defined in a second module.

edited Oct 16 '19 at 14:11

answered Oct 16 '19 at 13:29

Artiom Kozyrev

3,526
2
13
31

That doesn't change anything. Try `print(__name__)` in sleep. Process doesn't execute file, just function. – graphite Oct 16 '19 at 13:37
@graphite I did not write that process execute file but it should have access to the defined funtcion to do so the function should be either in separate file or be separated by name == main form other code, name == main define what happens when we run file directly, all above is "support" part. – Artiom Kozyrev Oct 16 '19 at 13:43
@graphite run OP code and you will get the following mistake: "RuntimeError: An attempt has been made to start a new process before the current process has finished its bootstrapping phase. This probably means that you are not using fork to start your child processes and you have forgotten to use the proper idiom in the main module: if __name__ == '__main__': freeze_support() The "freeze_support()" line can be omitted if the program is not going to be frozen to produce an executable." – Artiom Kozyrev Oct 16 '19 at 13:51
OP code runs for me as expected, and yes, it does fork. – graphite Oct 16 '19 at 13:53
@graphite I get the error which I provided in the message above and my code stops in 0,21 seconds after start. – Artiom Kozyrev Oct 16 '19 at 13:59
I belive you are right here, but still curious why it works for others. What OS and python version do you use? Did you change OP code? – graphite Oct 16 '19 at 14:05
@graphite I use Windows 10 Pro, Python 3.7 (C version) – Artiom Kozyrev Oct 16 '19 at 14:06
1

Got it. That is true for [spawn](https://docs.python.org/3/library/multiprocessing.html#the-spawn-and-forkserver-start-methods) which is default on Windows. – graphite Oct 16 '19 at 14:13
its does not change anything... output: ``` Finished in 0.16 second(s) ``` but thank you! – roye1233 Oct 16 '19 at 14:45
@roye1233 do you get in errors when you run the code? – Artiom Kozyrev Oct 16 '19 at 14:46
Artiom's code works as expected for me, taking just over 1 second to complete. – SyntaxVoid Oct 16 '19 at 14:51
@SyntaxVoid as was checked with graphite mp behavior differs in different OS, due to different mp module implementation for different OS. – Artiom Kozyrev Oct 16 '19 at 14:54
@roye1233 please specify your OS, python version and IDE (if you use any), probably it will clear why code work incorrectly for you. – Artiom Kozyrev Oct 16 '19 at 15:37

how to use multiprocessing in python right?

1 Answers1

Linked