I'm learning Python multiprocessing module and I've found this example:
from multiprocessing import Process, Queue
import time
def reader(queue):
## Read from the queue
while True:
msg = queue.get() # Read from the queue and do nothing
if (msg == 'DONE'):
break
def writer(count, queue):
## Write to the queue
for ii in xrange(0, count):
queue.put(ii) # Write 'count' numbers into the queue
queue.put('DONE')
if __name__=='__main__':
for count in [10**4, 10**5, 10**6]:
queue = Queue() # reader() reads from queue
# writer() writes to queue
reader_p = Process(target=reader, args=((queue),))
reader_p.daemon = True
reader_p.start() # Launch reader() as a separate python process
_start = time.time()
writer(count, queue) # Send a lot of stuff to reader()
reader_p.join() # Wait for the reader to finish
print "Sending %s numbers to Queue() took %s seconds" % (count,
(time.time() - _start))
I was wondering when the queue.get()
will return DONE
so I've tried following example:
#!/bin/env python
from multiprocessing import Process, Queue
import time
if __name__=='__main__':
queue = Queue()
print "Before 2x put"
queue.put(10)
queue.put(20)
print "After 2x put"
print "Before 1s get"
print queue.get()
print "After 1st get"
print "Before 2nd get"
print queue.get()
print "After 2nd get"
print "Before 3rd get"
print queue.get()
print "After 3rd get"
The last message from this script is Before 3rd get
, after this the script stuck and only way to end it is to terminate it. From this example you can see that queue.get()
is blocking (the code will not continue until it ends). How is it possible that in the original code queue.get()
returns DONE
, when this situation happens?
EDIT
In reply to @KemyLand which nicely explained what is going on here, this is the version that did not stuck:
#!/bin/env python
from multiprocessing import Process, Queue
import time
if __name__=='__main__':
queue = Queue()
print "Before 2x put"
queue.put(10)
queue.put(20)
print "After 2x put"
print "Before 1s get"
print queue.get()
print "After 1st get"
print "Before 2nd get"
print queue.get()
print "After 2nd get"
print "Before 3rd get"
try:
print queue.get_nowait()
print "After 3rd get"
except:
pass