Django: Maintaining a counter in a field (race condition)

Question

I have a model similar to this example

class Foo(models.Model):
    a = models.ForeignKey(...)
    number = models.IntegerField()

    @transaction.atomic
    def save(self, commit=True):
        if self.pk is None:            
            current_max = (
                Foo.objects
                .filter(a=self.a)
                .order_by('-number')
                .first()
            )
            current_max = 0 if current_max is None else current_max.number
            self.number = current_max + 1
        return super().save(commit)

The idea is that for each a, there will be a series of Foos numbered from 1 and then onwards.

The issue is that, even though we have @transaction.atomic, there is a race condition, because the transaction isolation level that Django expects will allow the transactions to run simultaneously, i.e.

A -> Get max -> 42
B -> Get max -> 42
A -> Set max + 1
A -> save
B -> Set max + 1
B -> save
Both will be 43

So how would I go about solving that? Is there a way to atomically setting the counter so I don't get the race condition between retrieving the current max and inserting the new value?

This question is similar to this one, but different enough that that question did not provide an answer to my specific example

Jason · Accepted Answer · 2018-07-25T11:04:00.773

4

Check out select_for_update. As described in the docs:

Returns a queryset that will lock rows until the end of the transaction, generating a SELECT ... FOR UPDATE SQL statement on supported databases.

Alternatively, there's two approaches defined by Haki Benata at https://web.archive.org/web/20170707121253/https://medium.com/@hakibenita/how-to-manage-concurrency-in-django-models-b240fed4ee2 which may be of interesting reading

Finally, if you need to lock the entire table, there's a method described that would allow you to do that. Effectively, you create a locking context manager that acquires a full table lock and releases it on exit

edited Jul 25 '18 at 11:04

answered Jul 25 '18 at 10:55

Jason

11,263
21
87
181

2

This locks a *row*. But here we need to lock the table, since at this moment, there is no row that maps to the instance, and the aggregate spans over the table. – Willem Van Onsem Jul 25 '18 at 10:58
You final idea, referencing the `LockedAtomicTransaction` from another answer is an appropriate solution for me, it seems. Thank you – Eldamir Jul 25 '18 at 12:25

Django: Maintaining a counter in a field (race condition)

1 Answers1