Sorry, I am a bit late to answer this question, but going by this comment, I extended the Queue in the multiprocessing package as per your requirements. Hopefully it will help someone in the future.
import multiprocessing as mp
from multiprocessing import queues
class IterQueue(queues.Queue):
def __init__(self, *args, **kwargs):
ctx = mp.get_context()
kwargs['ctx'] = ctx
super().__init__(*args, **kwargs)
# <---- Iter Protocol ------>
def __iter__(self):
return self
def __next__(self):
try:
if not self.empty():
return self.get() # block=True | default
else:
raise StopIteration
except ValueError: # the Queue is closed
raise StopIteration
Given below is a sample usage of this IterQueue
I wrote:
def sample_func(queue_ref):
for i in range(10):
queue_ref.put(i)
IQ = IterQueue()
p = mp.Process(target=sample_func, args=(IQ,))
p.start()
p.join()
print(list(IQ)) # [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
I have tested this IterQueue
for even a few more complex scenarios, and it seems to be working fine. Let me know if you think this works, or it could fail in some situation.