Responding to concurrent requests with Flask and eventlet

Question

I try to set up a minimal Flask application that uses eventlet to respond to concurrent requests instantly instead of blocking and responding to one request after the other (as the standard Flask debugging webserver does).

Prerequisites:

pip install Flask
pip install eventlet

From my understanding by what I found so far on the internet, it should work like this:

# activate eventlet
import eventlet
eventlet.monkey_patch()

from flask import Flask

import datetime
from time import sleep

# create a new Flask application
app = Flask(__name__)

# a short running task that returns immediately
@app.route('/shortTask')
def short_running_task():
  start = datetime.datetime.now()
  return 'Started at {0}, returned at {1}'.format(start, datetime.datetime.now())

# a long running tasks that returns after 30s
@app.route('/longTask')
def long_running_task():
  start = datetime.datetime.now()
  sleep(30)
  return 'Started at {0}, returned at {1}'.format(start, datetime.datetime.now())

# run the webserver
if __name__ == '__main__':
    app.run(debug=True)

When running this file, then opening http://localhost:5000/longTask in a webbrowser tab and while it is still processing opening another tab with http://localhost:5000/shortTask, I would expect the second tab to return immediately while the first tab is still loading. However, similar to when running this on the standard Werkzeug server, the second tab only returns just after the first one is finished after 30s.

What is wrong here? By the way, would this be what is commonly referred to as a "production ready webserver" for Flask, given that there are only few concurrent users to be expected (5 at most)?

By the way, when I use the Flask-socketio library to run the webserver which, according to the documentation, automatically chooses eventlet if it is installed, then it works as expected.

Complete example with Flask-socketio:

# activate eventlet
import eventlet
eventlet.monkey_patch()

from flask import Flask
from flask_socketio import SocketIO

import datetime
from time import sleep

# create a new Flask application
app = Flask(__name__)

# activate Flask-socketio
socketio = SocketIO(app)

# a short running task that returns immediately
@app.route('/shortTask')
def short_running_task():
  start = datetime.datetime.now()
  return 'Started at {0}, returned at {1}'.format(start, datetime.datetime.now())

# a long running tasks that returns after 30s
@app.route('/longTask')
def long_running_task():
  start = datetime.datetime.now()
  sleep(30)
  return 'Started at {0}, returned at {1}'.format(start, datetime.datetime.now())

# run the webserver with socketio
if __name__ == '__main__':
    socketio.run(app, debug=True)

Answer 1

When you run app.run(debug=True) you are explicitly telling Flask to run your application on the development web server, which is based on Werkzeug. It does not matter that you have loaded eventlet.

If you want to run your application on the eventlet web server, you have to start an eventlet web server, which according to the documentation is started as follows:

wsgi.server(eventlet.listen(('', 8000)), your_app)

This is more or less what socketio.run() does in my Flask-SocketIO extension, with a little bit more complexity to optionally handle SSL. The lines of code that do this are: https://github.com/miguelgrinberg/Flask-SocketIO/blob/539cd158f49ce085151911cb63edbacd0fa37173/flask_socketio/init.py#L391-L408. If you look around those lines, you will see that there are three different start up chunks of code, one for werkzeug, one for eventlet and one for gevent. They are all different.

Answer 2

import eventlet
eventlet.monkey_patch()

Won't magically turn your code into a multithread beast that can handle request asynchronously (it is still pretty magical and awesome).

As you can see in this example, you need to launch the wsgi server using eventlet wsgi's implementation.

If you want a standard solution, have a look at how to use nginx and uwsgi to launch flask application. You also may be interested in the project Spawning that leverage the pain of creating a complete multithread wsgi handler.