1

Let's say I have a replacement get_or_create method in line with this answer, except I need to combine this database transaction with an external API call (checking for the existence of an S3 bucket). Assume that there is a unique constraint at the database level (MySQL InnoDB). I'm running this code within multiple Gunicorn worker processes using eventlet.

@transaction.commit_on_success
def get_or_create_with_func(obj, func_to_call=None, **kwargs):
    try:
        result, created = obj.objects.create(**kwargs), True
    except IntegrityError:
        transaction.commit()
        result, created = obj.objects.get(**kwargs), False

    func_result = None
    if func_to_call is not None and created:
        func_result = func_to_call()

    return result, created, func_result

The success of the transaction is contingent upon func_to_call returning successfully (without raising an exception). If it raises an exception, obviously the transaction will rollback.

What I'm unsure of is if 2 threads are using this function at the same time, will create still save an entry to the database (so that IntegrityError is raised and the get call returns an object), or will it have to "wait" until func_to_call finishes?

Essentially I'm imagining this scenario, with the function above called like this:

query_result, created, func_result = get_or_create_with_func(MyModel, external_api_func, model_column=some_value)
  1. Thread 1 and 2 both call get_or_create_with_func at around the same time.
  2. They both try creating the entry via obj.objects.create
  3. Let's say thread 1 successfully creates it. It then exits the try/except block and then executes func_to_call.
  4. Thread 2 tries creating the object as well.
  5. Ideally, thread 2 will encounter an IntegrityError.
  6. Thread 1 finishes calling func_to_call, it does not raise an exception.
  7. Thread 2 "commits" (and Thread 1 has already finished committing as well), so it can call get, and gets the DB object written by Thread 1 as expected.

OR

  1. Thread 1 calls func_to_call and an exception is raised.
  2. Thread 1 causes django to rollback the transaction, so the DB entry does not exist.
  3. Thread 2 should hopefully still encounter an IntegrityError, but now calling get won't return any object since Thread 1 rolled back the transaction.

My question is, on step #4, will create fail due to the unique constraint on the table, or will it go through since Thread 1 hasn't actually committed the transaction yet (let's say func_to_call is fairly slow)?

Community
  • 1
  • 1
Nitzle
  • 2,417
  • 2
  • 22
  • 23

0 Answers0