1

Consider a Django application with a single RESTful API that creates objects (using Django REST Framework). As part of this API, I do some validation to make sure the creation calls are idempotent, such that if you call the creation API twice, the first will succeed, and the second will fail with a custom error code.

I have a scenario for testing this API which intermittently fails in the following way:

  1. First API call, success, returns 201 -> object has supposedly been created
  2. Immediately after response, second API call is made
  3. Validation logic calls MyModel.objects.get(some_field=some_value) to check if this is a duplicate call or not
  4. No such object is found, despite being created in step 1, thus a duplicate object is created
  5. When inspecting the admin/querying the model, both objects can be seen.

Some more data:

  • There is no explicit caching on this model, or any other caching involved in this process.
  • I am unable to reproduce this locally
  • on my deployment setup there is about a 5% failure rate for this possible race condition.
  • Both local and deployment use PostgreSQL.
  • Deployment environment does have general caching enabled, but when enabling cache locally still no repro.

What might be causing this race condition? Does Django ORM have any failure modes where I might be getting stale data? Is there any way I can defensively protect the validation from getting stale data?

Yuval Adam
  • 161,610
  • 92
  • 305
  • 395
  • 2
    I hesitate to ask this of someone who has 72,000 internet points, but would you show your code for steps 3-4? Also, are you sure that steps one and two are correct, in that the second call is started after the response has been returned from the first? – YPCrumble Sep 19 '15 at 02:15
  • You're trying to do an upsert or insert-if-not-exists using a check-and-insert pattern. This will not work. You need to attempt the insert, and trap the error if it fails. Or use PostgreSQL 9.5's upsert support. See http://stackoverflow.com/q/17267417/398670 – Craig Ringer Sep 19 '15 at 04:11
  • @YPCrumble the code here is irrelevant, I'm asking a very specific question about why a QuerySet might be returning stale data, and yes I've specifically mentioned that the first request has already returned a 201 response. – Yuval Adam Sep 19 '15 at 07:53
  • @CraigRinger I know what an upsert is and this isn't the case at all. I'm merely checking for the existence of an object so that I can return a customized error if it does exist. – Yuval Adam Sep 19 '15 at 07:54
  • 1
    @YuvalAdam insert-if-not-exists like you are facing the same race condition and same visibility issues as upsert. It's basically a degenerate case of upsert. – Craig Ringer Sep 19 '15 at 08:12

2 Answers2

0

Have a look to transaction.atomic:

https://docs.djangoproject.com/en/1.8/topics/db/transactions/#django.db.transaction.atomic

this sometimes solves such issue

DevLounge
  • 8,313
  • 3
  • 31
  • 44
0

My current proposed solution is based on the feedback from @CraigRinger which seems to be true. Basically, to get a consistent response from Postgres I need to actually attempt an INSERT, and not just query for the data, because there are race conditions in play.

A partial reference to this can be found in https://code.djangoproject.com/ticket/20429#comment:22

Bottom line, the solution is to add a DB-enforced unique=True constraint on the relevant field on the model (some_field in this case), attempt the object creation, catch the IntegrityError, and from there on I can implement the custom error handling and propagate the right result to the API layer.

Yuval Adam
  • 161,610
  • 92
  • 305
  • 395