Avoiding race conditions for a custom get_or_create in Django?

Question

Can anyone advise on the following problem:

I have a custom get_or_create method, which checks multiple fields and does some fancy stuff upon creation:

def fancy_get_or_create(name):
    object = self.fancy_get(name)
    if not object:
        object = self.fancy_create(name)
    return object

def fancy_get(name):
    return self.filter(Q(name=name) | Q(alias=name)).first()

def fancy_create(name):
    name = self.some_preprocessing(name)
    return self.create(name=name, alias=name)

There's a race condition, where one request will check to see if the object exists, find nothing, and start creating it. Before that request finishes creating the object, another request comes in looking for the same object, finds, nothing, and begins creating the new object. This request will fail because the database has some uniqueness constraints (the previous request had just created the object, so the second request will fail).

Is there any way to prevent request 2 from querying the database until request 1 has finished? I was reading about transaction management and it did not seem like the solution, since the issue is not partial updates (which would suggest an atomic transaction), but rather the need to make the second request wait until the first has finished.

Thanks!

Update: Here's what I went with:

try:
    return self.fancy_get(name) or self.fancy_create(name)
except IntegrityError:
    return self.fancy_get(name)

I don't consider this a duplicate, but this might give you some options: http://stackoverflow.com/questions/1030270/race-conditions-in-django?rq=1 — Foon, Feb 26 '15 at 17:33
I know this question has already been answered, but you should take at your database isolation levels to lock the database before reads are performed, basically creating a mutex on your get_or_create call. https://en.wikipedia.org/wiki/Isolation_(database_systems) — ESPmorph, Feb 27 '15 at 01:32

score 3 · Accepted Answer · answered Feb 26 '15 at 18:08

There are two viable solutions:

Use a mutex so only one process can access the fancy_get_or_create function at the same time.
Capture the error thrown by the database and do something instead: ignore that create, update the row instead of creating it, throw an exception, etc.

Edit: another solution might be doing an INSERT IGNORE instead of just an INSERT. https://dev.mysql.com/doc/refman/5.1/en/insert.html

Avoiding race conditions for a custom get_or_create in Django?

1 Answers1