Python synchronise between threads and processes

Question

A bit of background:

I am writing a function in Django to get the next invoice number, which needs to be sequential (not gaps), so the function looks like this:

def get_next_invoice_number():
    """
    Returns the max(invoice_number) + 1 from the payment records
    Does NOT pre-allocate number
    """
    # TODO ensure this is thread safe
    max_num = Payment.objects.aggregate(Max('invoice_number'))['invoice_number__max']
    if max_num is not None:
        return max_num + 1
    return PaymentConfig.min_invoice_number

Now the problem is, this function only returns the max()+1, in my production environment I have multiple Django processes, so if this function is called twice for 2 different payments (before the first record saved), they will get the same invoice number.

To mitigate this problem I can override the save() function to call the get_next_invoice_number() to minimise the time gap between these function calls, but there is still a very tiny chance for problem to happen.

So I want to implement a lock in the approve method, something like

from multiprocessing import Lock
lock = Lock()

class Payment(models.Model):
    def approve(self):
        lock.acquire()
        try:
            self.invoice_number = get_next_invoice_number()
            self.save()
        except: 
            pass
        finally:
            lock.release()

So my questions are:

Does this look okay?
The lock is for multiprocess, how about threads?

UPDATE:

As my colleague pointed out, this is not going to work when it's deployed to multiple servers, the locks will be meaningless.
Looks like DB transaction locking is the way to go.

1.) Why does it need to be sequential? 2.) Why can't you preallocate an invoice id upon the check (ie. using a DB's auto increment functionality)? No duplicates that way. — Andy, Sep 08 '14 at 22:18
@Andy it's a business accounting requirement for invoice number to be sequential. — James Lin, Sep 08 '14 at 22:19

score 0 · Answer 1 · edited May 23 '17 at 12:12

The easiest way to do this, by far, is with your database's existing tools for creating sequences. In fact, if you don't mind the value starting from 1 you can just use Django's AutoField.

If your business requirements are such that you need to choose a starting number, you'll have to see how to do this in the database. Here are some questions that might help.

Trying to ensure this with locks or transactions will be harder to do and slower to perform.

Python synchronise between threads and processes

1 Answers1