The parameter about python3 multiprocessing

Question

I am learn the multiprocessing, but when I want to create a process pool, I meet a question. My code is:

from multiprocessing import Pool, Queue

def read(q):
    print("flag")
    while not q.empty():
        value = q.get()
        print('Get %s from queue.' % value)

if __name__=='__main__':
    q = Queue()
    for i in range(4):
        q.put(i)
    p = Pool(4)
    for i in range(4):
        p.apply_async(read, args=(q,))
    p.close()
    p.join()
    print("down")

the child process seems not run, child process have no output. Result is

down

I guess it is related to parameters, so I changed code:

from multiprocessing import Pool, Queue

def read(i):
    print("flag")
    print(i)
    #while not q.empty():
        #value = q.get()
        #print('Get %s from queue.' % value)

if __name__=='__main__':
    q = Queue()
    for i in range(4):
        q.put(i)
    p = Pool(4)
    for i in range(4):
        p.apply_async(read, args=(i,))
    p.close()
    p.join()
    print("down")

Result is:

flag
0
flag
2
flag
1
flag
3
down

while process have output. So why the previous code can't run correct?

thanks for help.

score 1 · Answer 1 · answered Oct 28 '18 at 08:28

Your first problem is that you must call get() on each of the results from apply_async(), or nothing will happen. For example, try this:

if __name__=='__main__':
    q = Queue()
    for i in range(4):
        q.put(i)
    p = Pool(4)
    results = []
    for i in range(4):
        results.append(p.apply_async(read, args=(q,)))
    p.close()
    p.join()
    for result in results:
        print(result.get())
    print("down")

This will give you an error:

RuntimeError: Queue objects should only be shared between processes through inheritance

Your second problem is that you're not using Queue correctly--see here: Sharing a result queue among several processes

The parameter about python3 multiprocessing

1 Answers1