run code after transaction commit in Django

Question

Is there any way to run some code after transaction commit in Django?

I need to send some messages to a rabbitmq server for offline processing, but the message gets to the consumer before the Django transaction is commited.

My message is sent in the post_save signal of the model. What I'm looking for is a similar mechanism, using signals or something else, that would execute code after the commit (and do nothing if the transaction fails).

I haven't found any generic way of doing it in Django. Do you have any ideas?

I had a simmilar problem. On post_save, Publisher (process 1) saves task state and publishes the message. Consumer (process 2) receives the message and updates task state, which is not in the db yet. What worked was putting consumer to sleep for a second or two, after receiving the message. Feels dirty anyway. — ohnoes, Jan 19 '10 at 11:35

score 23 · Accepted Answer · edited May 01 '20 at 02:36

23

django-transaction-hooks solves this problem for Django < 1.9, and the functionality is built into Django 1.9+:

from django.db import transaction

def do_something():
    pass  # send a mail, invalidate a cache, fire off a Celery task, etc.

transaction.on_commit(do_something)

edited May 01 '20 at 02:36

David Wolever

148,955
89
346
502

answered Jun 04 '09 at 15:13

Carl Meyer

122,012
20
106
116

I have to work on some other parts of my app before, but I will follow this path. When it's ready, I'm going to drop an e-mail to django-developers and add a bug report with the patch. – Grégoire Cachet Jun 05 '09 at 14:16
If `django-transaction-hooks` requires too high a Django version, you can use `django-db-signals`. – r3m0t Jun 20 '14 at 12:41

score 15 · Answer 2 · answered Jan 12 '16 at 12:06

15

Hope this may help someone using Django 1.9 or later. Since 1.9 on_commit is available.

So basically you would be doing it like this:

from django.db import transaction

transaction.on_commit(
    lambda: send_msg_to_rabbitmqp(param1, param2, ...)
)

If you wish to keep post_save, you can still use on_commit:

@receiver(pre_save, sender=MyModel)
def my_handler(sender, instance, created, **kwargs):
    transaction.on_commit(
        lambda: send_msg_to_rabbitmqp(instance.id)
    )

answered Jan 12 '16 at 12:06

Igor Kramaric

173
2
5

Where should I put this code If I want the hooks to be executed every time on every transaction? – styrofoam fly Apr 10 '18 at 14:27
@styrofoamfly In the `save()` method of the model is where I put it. You can also register a `post_save` signal that calls `transaction.on_commit` and I think it get registered early enough to be called for that save transaction. – Greg Schmit May 11 '19 at 02:01
Could you elaborate on why we need the `lambda` syntax in order to pass parameters to the `on_commit` callable? Can `functools.partial` accomplish this in order to make it more obvious that we're calling a function with a full-definition elsewhere as opposed to a true, in-line lambda? – Addison Klinke Jul 13 '20 at 14:36
lambda is needed solely to turn `send_msg_to_rabbitmqp(instance.id)` into callable – Igor Kramaric Nov 09 '20 at 14:01

score 5 · Answer 3 · answered Dec 03 '09 at 03:15

5

I have implemented transaction signals (post_commit and post_rollback) by monkey patching django: http://gist.github.com/247844

answered Dec 03 '09 at 03:15

Grégoire Cachet

2,547
3
29
27

score 1 · Answer 4 · answered Dec 24 '12 at 06:32

1

Have a look at django-celery-transactions for a solution to this.

I've recently finished splitting-out and refactoring the underlying signals code code into a stand-alone app django-db-signals.

answered Dec 24 '12 at 06:32

bradley.ayers

37,165
14
93
99

score 1 · Answer 5 · answered Jun 04 '09 at 13:19

1

One possibility would be to subclass the transaction middleware so that it sends a custom signal on commit. Your code could listen for that signal, rather than post_save.

answered Jun 04 '09 at 13:19

Daniel Roseman

588,541
66
880
895

Thanks, I think that I will go with some middleware (I will probably add another middleware and not subclass the transaction middleware). I have a concern about signals though. Are they thread safe? If another thread throws a signal, may the current thread catch it? – Grégoire Cachet Jun 04 '09 at 13:43
I still have an issue with middlewares : if the application is running from a management command, it won't execute my callbacks. – Grégoire Cachet Jun 04 '09 at 14:02
I don't think your thread question really makes sense. Signals don't do anything special with regard to threads. A signal sent in one thread will call receivers in that same thread only. Built-in signal objects are module-global, however, so a signal handler registered for post_save in one thread is registered in all threads. (I think it might be possible to have signal objects in your own code that are not global, haven't looked at it carefully). – Carl Meyer Jun 04 '09 at 15:09

run code after transaction commit in Django

5 Answers5

Linked