Yield item from a thread

Question

from threading import Thread
import time
print 'start of script'

class MyThread(Thread):
    def __init__(self, start, end):
        self.start = start
        self.end = end
    def run(self):
        for i in xrange(self.start,self.end):
            yield i




my_threads = []

my_thread = MyThread(1,6)
my_thread.start()
my_threads.append(my_thread)

my_thread = MyThread(6,11)
my_thread.start()
my_threads.append(my_thread)

my_thread = MyThread(11,16)
my_thread.start()
my_threads.append(my_thread)


for t in my_threads:
    print t.join()

print 'end of script'

How can I do this correctly? I'm trying to print the numbers: range(1,16) where I"m getting this number from the output of a function run in a separate threads.

I understand that I won't get this range of numbers sequentially, as is the nature of functions run in separate threads.

I also know I can simply print them in the thread's function itself, but that's not the point, I'd like to print what I've yielded back in the main thread or main portion of my code.

score 5 · Answer 1 · edited May 23 '17 at 12:15

5

Threads don't return values, so you won't be able to yield values back to your main thread as you hope to. If you were to get your script to run (you would need to change the name of your start variable to something else since you're shadowing the start method), you would observe that the return value of t.join() is None. A common way to solve this is with a Queue, as was suggested in this similar question: Return value from thread

In your case, instead of calling yield i I would call queue.put(i) where queue was the Queue.Queue passed in during construction, then have a loop in the main thread before I joined my threads:

while True:
    try:
        print outqueue.get(True, 1)
    except Empty:
        break

for t in my_threads:
    print t.join()

That will wait up to 1 second for a new item before throwing Empty and breaking out of the while loop.

edited May 23 '17 at 12:15

Community

1
1

answered Jun 12 '11 at 20:40

Rob Marrowstone

1,224
8
15

It works.. however reason I want to use YIELD is to stay away from large variables, so I want to keep the q = Queue.Queue(2) and then I do q.get(True) but it lags and never prints? How can I keep the queue to only 2 items at a time and still get all the values I want? – MistahX Jun 12 '11 at 23:03
Well the `q.get(True)` will block forever, so when your generators are done, nothing will get printed and it will sit forever. Likewise initializing your `q` with `q = Queue.Queue(2)` will limit your `Queue` to just 2 objects, with your `q.put(i)` calls blocking until there is room for the put. I'm not sure exactly what you're asking, but another way to implement it would be to have the `Queue` block forever on the gets, but put something to indicate that your producers are done onto the `Queue` and have the main thread break out of the while loop when it sees that all the producers are done. – Rob Marrowstone Jun 13 '11 at 16:36
What happens for deamon threads that are not suppose to stop ? They will never `join`. You can fill a queue indefinitely, but how do you turn this queue to an iterator able to actually yield any new value ? In this case I don't want to break the loop, ever, so the idea would be to put it in a thread... and we go full circle – Eric Burel Mar 13 '18 at 16:45

score 0 · Answer 2 · answered Jun 12 '11 at 20:31

0

I think what you're looking for is a Queue. Pass a queue into your thread, then instead of yielding values, simply put them into the queue (myqueue.put(i)). Then you can get them in the main thread (myqueue.get()).

answered Jun 12 '11 at 20:31

Thomas K

39,200
7
84
86

Yield item from a thread

2 Answers2