Python multiprocessing module join() method does not prevent the interpreter from executing the code before the processes are finished

Question

Trying to play around with the multiprocessing module, and I used the join method, but it seems that it does not prevent the interpreter to wait for the processes to finish before it moves out to te rest of the script, here is my code (and you will understand better when you read the outpu):
import multiprocessing import time

start = time.perf_counter()
# creating simple sleep function
def sleeping(s):
    print(f'Sleeping for {s} second(s)...')
    time.sleep(s)
    print('Just woke up!')

# creating processes list
processes = []

# creating processes
if __name__ == '__main__':
    for _ in range(10):
        p = multiprocessing.Process(target=sleeping, args=(1,))
        p.start()
        processes.append(p)

for p in processes:
    p.join()


finish = time.perf_counter()

print(f'*****************Finished processing in {round(finish-start, 2)} second(s)')

Here is the output that I get:

*****************Finished processing in 0.0 second(s)
Sleeping for 1 second(s)...
*****************Finished processing in 0.0 second(s)
Sleeping for 1 second(s)...
*****************Finished processing in 0.0 second(s)
Sleeping for 1 second(s)...
*****************Finished processing in 0.0 second(s)
Sleeping for 1 second(s)...
*****************Finished processing in 0.0 second(s)
Sleeping for 1 second(s)...
*****************Finished processing in 0.0 second(s)
Sleeping for 1 second(s)...
*****************Finished processing in 0.0 second(s)
Sleeping for 1 second(s)...
*****************Finished processing in 0.0 second(s)
Sleeping for 1 second(s)...
*****************Finished processing in 0.0 second(s)
Sleeping for 1 second(s)...
*****************Finished processing in 0.0 second(s)
Sleeping for 1 second(s)...
Just woke up!
Just woke up!
Just woke up!Just woke up!

Just woke up!
Just woke up!
Just woke up!
Just woke up!Just woke up!

Just woke up!
*****************Finished processing in 1.12 second(s)

Also, I would like to understand why in some lines of the output it does not return to the next line and instead just prints the text next to the previous one.

I put the asterisks just to help me identify how many times the last line of my code has been executed. — Mehdi RH, Jun 08 '20 at 19:13
I don't understand what is not as expected? The `join` makes the main process wait until everybody _woke up_. That's exactly what `join` should do. The output can be mixed because all processes print in parallel to the same stream. — Dr. V, Jun 08 '20 at 19:16
possible duplicate https://stackoverflow.com/questions/25391025/what-exactly-is-python-multiprocessing-modules-join-method-doing — Nikos M., Jun 08 '20 at 19:22
@Dr.V the unexpected is that the last line ```print(f'*****************Finished processing in {round(finish-start, 2)} second(s)')``` is printed many times eventhough it is expected to be executed when every processeess is finished. — Mehdi RH, Jun 08 '20 at 19:42
Ah, now I understand ... sorry. It has nothing to do with `join`(not a duplicate of that one at least, as suggested by Nikos M). — Dr. V, Jun 08 '20 at 19:48

Dr. V · Answer 1 · 2020-06-08T19:52:33.987

Ah, sorry, now I see. it has nothing to do with join. When you start processes, the script must be prevented from being executed when loaded as a module, i.e. put everything into the

if __name__ == "__main__":
    ...

Otherwise, your script will be executed in every sub-process. Your print statement is outside.

Your script should look like this:

import time
import multiprocessing

# creating simple sleep function
def sleeping(s):
    print(f'Sleeping for {s} second(s)...')
    time.sleep(s)
    print('Just woke up!')

def main():
    start = time.perf_counter()

    # creating processes list
    processes = []

    for _ in range(10):
        p = multiprocessing.Process(target=sleeping, args=(1,))
        p.start()
        processes.append(p)

    for p in processes:
        p.join()


    finish = time.perf_counter()

    print(f'*****************Finished processing in {round(finish-start, 2)} second(s)')

if __name__ == "__main__":
    main()

Python multiprocessing module join() method does not prevent the interpreter from executing the code before the processes are finished

1 Answers1