Flask session doesn't update consistently with parallel requests

Question

I'm noticing that when requests running in parallel modify Flask's session, only some keys are recorded. This happens both with Flask's default cookie session and with Flask-Session using the Redis backend. The project is not new, but this only became noticeable once many requests were happening at the same time for the same session.

import time
from flask import Flask, session
from flask_session import Session

app = Flask(__name__)
app.secret_key = "example"
app.config["SESSION_TYPE"] = "redis"
Session(app)

@app.route("/set/<value>")
def set_value(value):
    """Simulate long running task."""
    time.sleep(1)
    session[value] = "done"
    return "ok\n"

@app.route("/keys")
def keys():
    return str(session.keys()) + "\n"

The following shell script demonstrates the issue. Notice that all the requests complete, but only one key is present in the final listing, and it's different between test runs.

#!/bin/bash
# set session
curl -c 'cookie' http://localhost:5007/keys
# run parallel
curl -b 'cookie' http://localhost:5007/set/key1 && echo "done1" &
curl -b 'cookie' http://localhost:5007/set/key2 && echo "done2" & 
curl -b 'cookie' http://localhost:5007/set/key3 && echo "done3" &
wait
# get result
curl -b 'cookie' http://localhost:5007/keys

$ sh test.sh 
dict_keys(['_permanent'])
ok
ok
ok
done3
done1
done2
dict_keys(['_permanent', 'key2'])

$ sh test.sh 
dict_keys(['_permanent'])
ok
done3
ok
ok
done2
done1
dict_keys(['_permanent', 'key1'])

Why aren't all the keys present after the requests finish?

davidism · Accepted Answer · 2018-10-19T15:49:11.910

Cookie-based sessions are not thread safe. Any given request only sees the session cookie sent with it, and only returns the cookie with that request's modifications. This isn't specific to Flask, it's how HTTP requests work.

You issue three requests in parallel. They all read the initial cookie that only contains the _permanent key, send their requests, and get a response that sets a cookie with their specific key. Each response cookie would have the _permanent key and the key_keyN key only. Whichever request finishes last writes to the file, overwriting previous data, so you're left with its cookie only.

In practice this isn't an issue. The session isn't really meant to store data that changes rapidly between requests, that's what a database is for. Things that modify the session, such as logging in, don't happen in parallel to the same session (and are idempotent anyway).

If you're really concerned about this, use a server-side session to store the data in a database. Databases are good at synchronizing writes.

You're already using Flask-Session and Redis, but digging into the Flask-Session implementation reveals why you have this issue. Flask-Session doesn't store each session key separately, it writes a single serialized value with all the keys. So it suffers the same issue as cookie-based sessions: only what was present during that request is put back into Redis, overwriting what happened in parallel.

In this case, it will be better to write your own SessionInterface subclass to store each key individually. You would override save_session to set all keys in session and delete any that aren't present.

Flask session doesn't update consistently with parallel requests

1 Answers1