How does the python multiprocessing works in backend?

Question

When i tried to run the code:

import multiprocessing

def worker():
    """worker function"""
    print 'Worker'
    return

if __name__ == '__main__':
    jobs = []
    for i in range(5):
        p = multiprocessing.Process(target=worker)
        jobs.append(p)
        p.start()

The output is blank and simply executing without printing "Worker". How to print the required output in multiprocessing?
What actually is happening while using multiprocessing?
What is the maximum number of cores we can use for multiprocessing?

On my PC, your test prints "worker" five times. What OS are you using? — Robᵩ, May 23 '16 at 15:37
Did you run this as a script, or in interactive mode? What happens if you `join` the processes after you've started all of them? — user2357112, May 23 '16 at 15:40

notorious.no · Answer 1 · 2016-05-23T15:53:40.320

1

I've tried your code in Windows 7, Cygwin, and Ubuntu. For me all the threads finish before the loop comes to an end so I get all the prints to show, but using join() will guarantee all the threads will finish.

import multiprocessing

def worker():
    """worker function"""
    print 'Worker'
    return

if __name__ == '__main__':
    jobs = []
    for i in range(5):
        p = multiprocessing.Process(target=worker)
        jobs.append(p)
        p.start()

    for i in range(len(jobs)):
        jobs.pop().join()

As far as how multiprocessing works in the backend, I'm going to let someone more experienced than myself answer that one :) I'll probably just make a fool of myself.

edited May 23 '16 at 15:53

answered May 23 '16 at 15:47

notorious.no

4,919
3
20
34

`join` is not necessary unless you make child processes daemonic. `multiprocessing` will join non-deamon processes when main script exits. – robyschek May 23 '16 at 15:59
Yes you're absolutely correct, ``join`` isn't required, but the original code works for me and others, so maybe Win10 does something strange with processes. My "answer" was more of a comment :) – notorious.no May 23 '16 at 16:05
1

@robyschek I don't see this in the docs, is this always guaranteed to be true? – Bi Rico May 23 '16 at 16:06
@BiRico Well, [The doc](https://docs.python.org/2/library/multiprocessing.html#all-platforms) states `Remember also that non-daemonic processes will be joined automatically`. It does it however in the guidlines section. But for me it always worked, so I thought it is fully documented. – robyschek May 23 '16 at 16:15

score 0 · Answer 2 · answered May 23 '16 at 15:38

0

I get 5 time "Worker" printed for my part, are you on Python 3 ? if it is the case you muste use print("Worker"). from my experiment, I think multitreading doesn't mean using multiple cores, it just run the diferent tread alternatively to ensure a parallelism. try reading the multiprocessing lib documentation for more info.

answered May 23 '16 at 15:38

user5698387

477
3
10

i'm using python 2.7.9 i mentioned multiprocessing not multi-threading. In multiprocessing documentation it was given how to use the module but not how it works in backend – dinece May 23 '16 at 15:41
it may inform you http://stackoverflow.com/questions/3044580/multiprocessing-vs-threading-python i am not sure why you don't get it printed. – user5698387 May 23 '16 at 15:45
i tryed it whith more than my number of core and it unsuprisingly printed well again – user5698387 May 23 '16 at 15:50
even i tried in Python 3, no output is printed instead i got RESTART: C:/Users/les/AppData/Local/Programs/Python/Python35-32/dinesh/test-1.py – dinece May 24 '16 at 10:22

How does the python multiprocessing works in backend?

2 Answers2