why does my print function (in multiprocessing) print nothing?

Question

why does my print function (in python multiprocess) print nothing?

from multiprocessing import Process, Queue
import os, time, random


def write(q):
    print('Process to write: %s' % os.getpid())
    for value in ['A', 'B', 'C']:
        print('Put %s to queue...' % value)
        q.put(value)
        time.sleep(random.random())


def read(q):
    print('Process to read: %s' % os.getpid())
    while True:
        value = q.get(True)
        print('Get %s from queue.' % value)

if __name__=='__main__':

    q = Queue()
    pw = Process(target=write, args=(q,))
    pr = Process(target=read, args=(q,))

    pw.start()
    print('start')

    pr.start()

    pw.join()

    pr.terminate()
    print('end')

I run it on spyder (windows 10 system).

My result on IPython console of Spyder:

runfile('C:/Users/Dust/Desktop/programs/crawl/test.py', wdir='C:/Users/Dust/Desktop/programs/crawl')
start
end

Result on Python console of Spyder:

>>> runfile('C:/Users/Dust/Desktop/programs/crawl/test.py', wdir='C:/Users/Dust/Desktop/programs/crawl')
start
Process to write: 12824
Put A to queue...
Put B to queue...
Put C to queue...
end

It is really weird. The results are different, but both are not what I want.

Could anyone help me find where is the problem in my program. Thanks a lot

Can you share some more information? Does none of your `print()` statements produce any output? Does your code terminate without any errors? — Claudio Brasser, Aug 09 '17 at 09:48
I've had trouble with print statements in threads before. Have you tried `print('Process to write: %s' % os.getpid(), flush=True)`? — orangeInk, Aug 09 '17 at 09:52
it prints ok like this: `start, Process to write: 3956, Put A to queue..., Process to read: 4144, Get A from queue., Put B to queue..., Get B from queue., Put C to queue..., Get C from queue., end` Whats your problem. are you using terminal? — Rahul, Aug 09 '17 at 09:56
My result only has "start" and "end" which are in the mainprocess. And there is no error information. P.S.　I run it on IPython console of Spyder (win 10). — D.., Aug 09 '17 at 14:22
The `subprocess` and `multiprocessing` modules are two different things. Conflating them (as using the former in the title when the question is about the latter) is not helpful. — Charles Duffy, Aug 09 '17 at 14:52
@D.., have you evaluated whether the answer given in the proposed duplicate fixes your problem? — Charles Duffy, Aug 09 '17 at 14:54
@Xingzhou Liu, it does work on the Python console of Spyder if I and **sys.stdout.flush()** after print function. But it still doesn't work on IPython console of Spyder. — D.., Aug 09 '17 at 15:15
You wait for `write` to put things into the queue. You don't wait for `read` to take anything out. — user2357112, Aug 09 '17 at 18:20
@orangeInk Did you run your program on IPython？It still doesn't work. Thanks anyway. — D.., Aug 10 '17 at 01:58
@user2357112 Do you mean I have a `time.sleep()` function in `write` , but `read` has none? I've tried it. It doesn't work... — D.., Aug 10 '17 at 02:03

score 1 · Answer 1 · answered Jan 09 '20 at 18:00

The multiprocessing module uses fork to spawn child processes for parallelism. Windows does not implement fork, and Python emulation of it on Windows is incomplete. One of the missing effects is that stdout (used by print()) is not inherited by the child processes. Rather, child processes use the default stdout.

This works fine in a command window because it uses default stdout. This shows nothing in Spyder because a different pipe is used for output to the Spyder IPython console window.

See this question ("Simple Multiprocessing function in Spyder doesn't output results") and this answer for additional details and potential mitigations.

why does my print function (in multiprocessing) print nothing?

1 Answers1

Linked