Django tests: how to test concurrent users on SQLite?

Question

My Django webapps are multi-player strategy games that involve a player proceeding through a sequence of pages and submitting a ModelForm on each page. They are usually back-and-forth games, so Player 1 makes a decision while Player 2 sees a waiting screen, then vice versa, etc. The games can have a variable number of players.

I have implemented tests using Django's HTTP test client, where the tests are designed to be written from the perspective of an individual user, with conditional logic that branches whether the client plays as P1 or P2, and with the waiting screen abstracted away:

def play(self):

    self.submit(views.Introduction)
    self.submit(views.Question1, {'answer': random.choice([True,False])})

    if self.player.id_in_group == 1:
        self.submit(views.Send, {"sent_amount": 4})
    else:
        self.submit(views.SendBack, {'sent_back_amount': 8})

    self.submit(views.Results)
    self.submit(views.Question2, dict(feedback=4))

Then, to simulate N players, I instantiate N threads, each executing the code above.

This approach works on Postgres, but not with SQLite, which for external reasons is the engine we need to use for local development. I get: OperationalError: Database is locked

Is there some way to avoid the locks? Like a task queue, or sleeping until the SQlite DB is unlocked? Or somehow looping through the test clients in random order and executing a few lines from each at a given time? It's OK if we slow down the performance a bit on SQLite.

Note: I could write the code of multiple players inline so that they all execute in 1 thread, but (1) I find that code harder to read, (2) it is artificial because it hardcodes an exact order in which actions will be executed, and (3) it doesn't scale well when, say, we change a 10-player game to a 20-player game.

https://docs.djangoproject.com/en/dev/ref/databases/#database-is-locked-errors The problem with sqlite is that it's "lite", it's very limited when it comes to concurrency. I understand that you say sqlite is required, but really the only actual workaround would be to have each developer setup a local postrges server for development. This can be easily done with tools like vagrant or buildout. — Ngenator, Oct 24 '14 at 18:32
possible duplicate of [OperationalError: database is locked](http://stackoverflow.com/questions/3172929/operationalerror-database-is-locked) — Louis, Oct 24 '14 at 22:26

score 1 · Answer 1 · answered Oct 25 '14 at 01:50

SQLite supports multi threaded mode. In this mode, SQLite can be safely used by multiple threads provided that no single database connection is used simultaneously in two or more threads.

Celery provides a way to ensure only one task runs at a time. If you celery for database transactions, then celery makes sure to allow only one transaction to run at a time, which solves your problem.

Django tests: how to test concurrent users on SQLite?

1 Answers1