Keep alive connection between two "request/response" services

Question

we have a Django app running on servers in Central Europe, a service provider who has servers in US and proxy servers with a simple Flask app in the US. The Flask app takes requests from the provider and forwards them to our main Django app. We can't run the Django app in the US.

Currently, we run into a problem that the request (with response) pretty often takes a little bit longer than it can take - it has to be under 3 seconds - partially because of the heavy load on our main Django app, so we want to optimize the flow.

Right now the communication between the Flask and Django app is a simple Requests.post for each request from the provider which opens a new TCP connection. This can be a little bit time consuming between the US and Europe (because of the handshake).

What I was thinking is to create some sort of a "permanent" connection between the two apps, which would not require the handshake for each request and thus saving us a little bit of time. The naive solution is to create an HTTP server on the Django side and some service (with multiple workers) on the Flask side which will create connections, keep them alive, listen to the Flask proxy and forward the requests from the Flask proxy through already opened TCP connections to the Django app.

It seems a little as overkill but also as something, that has to be already implemented. Do you have any suggestions on how to solve this problem as elegantly as possible, without touching the Flask or Django apps (only to change URLs...)? Thanks for your thoughts

Charlestone · Accepted Answer · 2020-06-29T07:19:59.717

In the end, we created a requests.Session for each thread and stored it to a threading.local object in the Flask app. With this approach, each worker should keep its own session alive. This works, because the wgsi app does not create a new thread for each request, but reuses existing threads from its thread pool.

thread_data = threading.local()

def get_local_session() -> requests.Session:
    """
    Returns session for a current thread. If no session exists yet, a new session is created.
    """
    if not hasattr(thread_data, 'session'):
        thread_data.session = requests.session()
    return thread_data.session

response = get_local_session().post(URL, data=req_body, headers={'Content-Type': req_content_type}, timeout=REQUEST_TIMEOUT)

We had to change keepalive_timeout setting in nginx (which sits in front of the django app) to keep these connections open for long times.

Keep alive connection between two "request/response" services

1 Answers1