No output from Process using multiprocessing

Question

I am a beginner in multiprocessing, can anyone tell me why this does not produce any output?

import multiprocessing

def worker(num):
    """thread worker function"""

    print('Worker:', num)

if __name__ == '__main__':
    jobs = []

    for i in range(4):
        p = multiprocessing.Process(target=worker, args=(i,))
        jobs.append(p)
        p.start()

Do you run your code interactively in or as a script (eg `python myscript.py`) ? — robyschek, Jun 01 '16 at 17:36
It seems you have two problems! One is not calling `join()` and the other is an IDLE-specific issue. See my edit to my answer. Hope this helps! — Will, Jun 01 '16 at 18:46

score 3 · Answer 1 · edited May 23 '17 at 10:28

You're starting your Process(), but never waiting on it to complete, so your program's execution ends before the background process finishes. Try this, with a call to Process.join():

import multiprocessing
import sys

def worker(num):
    """thread worker function"""

    print('Worker:', num)
    sys.stdout.flush()

if __name__ == '__main__':
    jobs = []

    for i in range(4):
        p = multiprocessing.Process(target=worker, args=(i,))
        jobs.append(p)
        p.start()

    map(lambda p: p.join(), jobs)

Here we use map() to call join() on each process in the jobs list.

Here's an example of this working:

In [127]: %paste
import multiprocessing
import sys

def worker(num):
    """thread worker function"""

    print('Worker:', num)
    sys.stdout.flush()

if __name__ == '__main__':
    jobs = []

    for i in range(4):
        p = multiprocessing.Process(target=worker, args=(i,))
        jobs.append(p)
        p.start()

    map(lambda p: p.join(), jobs)

## -- End pasted text --
Worker: 2
Worker: 1
Worker: 0
Worker: 3

In [128]:

As for why this isn't working in IDLE, see this answer:

However, to see the print output, at least on Windows, one must start IDLE from a console like so.
C:\Users\Terry>python -m idlelib
hello bob
(Use idlelib.idle on 2.x.) The reason is that IDLE runs user code in a separate process. Currently the connection between the IDLE process and the user code process is via a socket. The fork done by multiprocessing does not duplicate or inherit the socket connection. When IDLE is started via an icon or Explorer (in Windows), there is nowhere for the print output to go. When started from a console with python (rather than pythonw), output goes to the console, as above.

Weird thing. In ipython when I run the **code with join** after the **code without join** the missing messages from the **code without join** are also flushed. So this is not about premature exit, but smth about stdout caching in ipython. — robyschek, Jun 01 '16 at 17:57
@user128751 I just verified that if you start IDLE from the console, as I said in the quoted text, Will's revision with join work's when run from an IDLE editor. — Terry Jan Reedy, Jun 01 '16 at 21:13

score 2 · Answer 2 · answered Jun 01 '16 at 17:29

2

You probably need to flush the output. You can use sys.stdout.flush():

import multiprocessing
import sys

def worker(num):
    """thread worker function"""
    print('Worker:', num)
    sys.stdout.flush()
    return

if __name__ == '__main__':
    jobs = []
    for i in range(4):
        p = multiprocessing.Process(target=worker, args=(i,))
        jobs.append(p)
        p.start()

answered Jun 01 '16 at 17:29

rofls

4,993
3
27
37

2

This did not help! – user128751 Jun 01 '16 at 17:31
You can try putting it in the for loop or afterwards -- for me it works without flush. – rofls Jun 01 '16 at 17:34
1

@user128751 how are you running the program? Terminal, interpreter, IDE? – rofls Jun 01 '16 at 17:34

score 0 · Answer 3 · answered Jul 24 '23 at 14:30

The current explanation in the IDLE doc section "Running User Code":

By default, IDLE runs user code in a separate OS process rather than in the user interface process that runs the shell and editor. In the execution process, it replaces sys.stdin, sys.stdout, and sys.stderr with objects that get input from and send output to the Shell window. The original values stored in sys.stdin, sys.stdout, and sys.stderr are not touched, but may be None.

...

IDLE’s standard stream replacements are not inherited by subprocesses created in the execution process, whether directly by user code or by modules such as multiprocessing. If such subprocess use input from sys.stdin or print or write to sys.stdout or sys.stderr, IDLE should be started in a command line window. (On Windows, use python or py rather than pythonw or pyw.) The secondary subprocess will then be attached to that window for input and output.

user128751: I started IDLE in Windows Command Prompt with py -m idlelib. Then I pasted your code as is into Editor and ran it. The expected output appeared in Command Prompt, after the command line.

Worker: 0
Worker: 1
Worker: 2
Worker: 3

Other GUI IDEs will tend to have the same issue.

No output from Process using multiprocessing

3 Answers3

Linked