Python thread never starts if run() contains yield from

Question

Python 3.4, I'm trying to make a server using the websockets module (I was previously using regular sockets but wanted to make a javascript client) when I ran into an issue (because it expects async, at least if the examples are to be trusted, which I didn't use before). Threading simply does not work. If I run the following code, bar will never be printed, whereas if I comment out the line with yield from, it works as expected. So yield is probably doing something I don't quite understand, but why is it never even executed? Should I install python 3.5?

import threading

class SampleThread(threading.Thread):
    def __init__(self):
        super(SampleThread, self).__init__()
        print("foo")

    def run(self):
        print("bar")
        yield from var2

thread = SampleThread()
thread.start()

you cannot use `yield` from a run routine which is supposed to loop forever / do something, not return/yield something. — Jean-François Fabre, Dec 17 '16 at 23:17
my main question was, why? But it looks like the answer lies in that yield automatically turns the entire function into a generator creator so to speak. But how can I use asyncio together with threading, or is it silly to do so? I originally had an thread for each client to see if they had anything to receive, but can I just turn them into normal objects if I use async? — fdagpigj, Dec 17 '16 at 23:28
Yes, it's silly way in 99% cases. Usually async program should have the single event loop in main thread + thread pool accessible by `loop.run_in_executor()`. But if you really need it -- start a new event loop explicitly in your thread. — Andrew Svetlov, Dec 18 '16 at 17:05

score 0 · Answer 1 · edited May 23 '17 at 12:30

This is not the correct way to handle multithreading. run is neither a generator nor a coroutine. It should be noted that the asyncio event loop is only defined for the main thread. Any call to asyncio.get_event_loop() in a new thread (without first setting it with asyncio.set_event_loop() will throw an exception.

Before looking at running the event loop in a new thread, you should first analyze to see if you really need the event loop running in its own thread. It has a built-in thread pool executor at: loop.run_in_executor(). This will take a pool from concurrent.futures (either a ThreadPoolExecutor or a ProcessPoolExecutor) and provides a non-blocking way of running processes and threads directly from the loop object. As such, these can be await-ed (with Python3.5 syntax)

That being said, if you want to run your event loop from another thread, you can do it thustly:

import asyncio

class LoopThread(threading.Thread):
    def __init__(self):
        self.loop = asyncio.new_event_loop()

    def run():
        ayncio.set_event_loop(self.loop)
        self.loop.run_forever()

    def stop():
        self.loop.call_soon_threadsafe(self.loop.stop)

From here, you still need to device a thread-safe way of creating tasks, etc. Some of the code in this thread is usable, although I did not have a lot of success with it: python asyncio, how to create and cancel tasks from another thread

Python thread never starts if run() contains yield from

1 Answers1