Right way to "timeout" a Request in Tornado

Question

I managed to code a rather silly bug that would make one of my request handlers run a very slow DB query.

Interesting bit is that I noticed that even long-after siege completed Tornado was still churning through requests (sometimes 90s later). (Comment --> I'm not 100% sure of the workings of Siege, but I'm fairly sure it closed the connection..)

My question in two parts: - Does Tornado cancel request handlers when client closes the connection? - Is there a way to timeout request handlers in Tornado?

I read through the code and can't seem to find anything. Even though my request handlers are running asynchronously in the above bug the number of pending requests piled up to a level where it was slowing down the app and it would have been better to close out the connections.

score 6 · Accepted Answer · answered Aug 15 '14 at 14:44

Tornado does not automatically close the request handler when the client drops the connection. However, you can override on_connection_close to be alerted when the client drops, which would allow you to cancel the connection on your end. A context manager (or a decorator) could be used to handle setting a timeout for handling the request; use tornado.ioloop.IOLoop.add_timeout to schedule some method that times out the request to run after timeout as part of the __enter__ of the context manager, and then cancel that callback in the __exit__ block of the context manager. Here's an example demonstrating both of those ideas:

import time
import contextlib

from tornado.ioloop import IOLoop
import tornado.web
from tornado import gen

@gen.coroutine
def async_sleep(timeout):
    yield gen.Task(IOLoop.instance().add_timeout, time.time() + timeout)

@contextlib.contextmanager
def auto_timeout(self, timeout=2): # Seconds
    handle = IOLoop.instance().add_timeout(time.time() + timeout, self.timed_out)
    try:
        yield handle
    except Exception as e:
        print("Caught %s" % e)
    finally:
        IOLoop.instance().remove_timeout(handle)
        if not self._timed_out:
            self.finish()
        else:
            raise Exception("Request timed out") # Don't continue on passed this point

class TimeoutableHandler(tornado.web.RequestHandler):
    def initialize(self):
        self._timed_out = False

    def timed_out(self):
        self._timed_out = True
        self.write("Request timed out!\n")
        self.finish()  # Connection to client closes here.
        # You might want to do other clean up here.

class MainHandler(TimeoutableHandler):

    @gen.coroutine
    def get(self):
        with auto_timeout(self): # We'll timeout after 2 seconds spent in this block.
            self.sleeper = async_sleep(5)
            yield self.sleeper
        print("writing")  # get will abort before we reach here if we timed out.
        self.write("hey\n")

    def on_connection_close(self):
        # This isn't the greatest way to cancel a future, since it will not actually
        # stop the work being done asynchronously. You'll need to cancel that some
        # other way. Should be pretty straightforward with a DB connection (close
        # the cursor/connection, maybe?)
        self.sleeper.set_exception(Exception("cancelled"))


application = tornado.web.Application([
    (r"/test", MainHandler),
])
application.listen(8888)
IOLoop.instance().start()

any idea why I get this problem: ```TypeError: object generator can't be used in 'await' expression``` — lateautumntear, Jul 15 '21 at 11:28
@lateautumntear This answer pre-dates async/await syntax, so it may not work as-is with new versions of tornado. Probably you need to use an async method instead of one decorated with `@gen.coroutine`, and maybe replace the yields with awaits? — dano, Jul 15 '21 at 12:18

score 0 · Answer 2 · answered Sep 10 '19 at 17:47

Another solution to this problem is to use gen.with_timeout:

import time
from tornado import gen
from tornado.util import TimeoutError


class MainHandler

    @gen.coroutine
    def get(self):
        try:
            # I'm using gen.sleep here but you can use any future in this place
            yield gen.with_timeout(time.time() + 2, gen.sleep(5))
            self.write("This will never be reached!!")
        except TimeoutError as te:
            logger.warning(te.__repr__())
            self.timed_out()

    def timed_out(self):
        self.write("Request timed out!\n")

I liked the way handled by the contextlib solution but I'm was always getting logging leftovers.

The native coroutine solution would be:

async def get(self):
    try:
        await gen.with_timeout(time.time() + 2, gen.sleep(5))
        self.write("This will never be reached!!")
    except TimeoutError as te:
        logger.warning(te.__repr__())
        self.timed_out()

Right way to "timeout" a Request in Tornado

2 Answers2

Linked