3

Consider (where queue is an asyncio.Queue, and I'm using the asyncio websockets library which uses the same server interface as asyncio in the standard library):

async def handler(websocket: websockets.WebSocketServerProtocol, path):
    while True:
        data = await queue.get()
        await websocket.send(data)

What I'm finding is that the ConnectionClosed exception (triggered either when a client disconnects, and is also used for server shutdown) is only raised, obviously, within the loop when we are awaiting on send(). This can happen long after a websocket client has disconnected, because we will spend most of the time awaiting on queue.get().

Furthermore, I'm trying to implement graceful shutdown of the server. That code looks very similar to the standard

server.close()
loop.run_until_complete(server.wait_closed())

The issue is that my websocket handler, i.e., handler(), nevertheless still continues to wait on the queue for more data before raising the exceptions that wait_closed() placed on the event loop.

My question: is it correct to introduce the use of wait_for() to make the handler() function more responsive to these management events:

async def handler(websocket: websockets.WebSocketServerProtocol, path):
    while True:
        try:
            data = await loop.wait_for(queue.get(), 5.0)  # 5 seconds
            await websocket.send(data)
        except asyncio.TimeoutError:
            logging.debug('Refreshing the loop')

Most of the asyncio examples I've found online have been quite simplistic and don't seem to cover these kinds of issues.

I could also put junk on the queue during my shutdown sequence, which would allow the exception to be raised, but that wouldn't help catching client disconnections earlier.

Any good references with examples would be much appreciated. I'm finding the official asyncio docs somewhat terse.

Caleb Hattingh
  • 9,005
  • 2
  • 31
  • 44
  • This might be helpfull http://stackoverflow.com/questions/35604033/waiting-for-a-task-to-complete-after-keyboardinterrupt-in-asyncio The answer is more generic than the question. – brunsgaard Feb 26 '16 at 02:20
  • 2
    To answer short, `await queue.get()` is doomed, you application will throw exception on shutdown. And queue.get() with timeout, is mem leaking atm http://bugs.python.org/issue26259. Use queue.get_nowait() with try/except. That is the way to go ;) I am working on a patch for the mem leak problem at the moment. – brunsgaard Feb 26 '16 at 02:24
  • Thank you for your comment! That memory issue is very interesting and something I definitely hadn't considered. If you would be willing to create an answer showing the code for how to use `get_nowait()` in an idiomatic way (I think it would help others reading this) I'll accept your answer – Caleb Hattingh Feb 26 '16 at 02:48
  • Now i read my comment again, of cause it will not throw exception, but it will tell you that you have pending coroutines. ;) – brunsgaard Feb 26 '16 at 03:02
  • Did you read the other answer i wrote, which I linked to in the first comment? – brunsgaard Feb 26 '16 at 03:03
  • Yes. If you're busy that's fine, I can write an answer here more specific to my question. Thanks again for your help. – Caleb Hattingh Feb 26 '16 at 03:37

0 Answers0