Let's say I have a replacement get_or_create
method in line with this answer, except I need to combine this database transaction with an external API call (checking for the existence of an S3 bucket). Assume that there is a unique constraint at the database level (MySQL InnoDB). I'm running this code within multiple Gunicorn worker processes using eventlet.
@transaction.commit_on_success
def get_or_create_with_func(obj, func_to_call=None, **kwargs):
try:
result, created = obj.objects.create(**kwargs), True
except IntegrityError:
transaction.commit()
result, created = obj.objects.get(**kwargs), False
func_result = None
if func_to_call is not None and created:
func_result = func_to_call()
return result, created, func_result
The success of the transaction is contingent upon func_to_call
returning successfully (without raising an exception). If it raises an exception, obviously the transaction will rollback.
What I'm unsure of is if 2 threads are using this function at the same time, will create
still save an entry to the database (so that IntegrityError
is raised and the get
call returns an object), or will it have to "wait" until func_to_call
finishes?
Essentially I'm imagining this scenario, with the function above called like this:
query_result, created, func_result = get_or_create_with_func(MyModel, external_api_func, model_column=some_value)
- Thread 1 and 2 both call
get_or_create_with_func
at around the same time. - They both try creating the entry via
obj.objects.create
- Let's say thread 1 successfully creates it. It then exits the try/except block and then executes
func_to_call
. - Thread 2 tries creating the object as well.
- Ideally, thread 2 will encounter an
IntegrityError
. - Thread 1 finishes calling
func_to_call
, it does not raise an exception. - Thread 2 "commits" (and Thread 1 has already finished committing as well), so it can call
get
, and gets the DB object written by Thread 1 as expected.
OR
- Thread 1 calls
func_to_call
and an exception is raised. - Thread 1 causes django to rollback the transaction, so the DB entry does not exist.
- Thread 2 should hopefully still encounter an
IntegrityError
, but now callingget
won't return any object since Thread 1 rolled back the transaction.
My question is, on step #4, will create
fail due to the unique constraint on the table, or will it go through since Thread 1 hasn't actually committed the transaction yet (let's say func_to_call
is fairly slow)?