Avoiding redundant write operations with Django ORM

Question

I have a periodically running task that performs a lot of write calls to the database on existing data. In most cases, records that are written don't contain a change, so the update queries are redundant. The redundant write calls do put an unnecessarily high load on our database replication, though.

This is roughly the algorithm:

Fetch existing row from database
Potentially modify a lot of fields based on parameters that were passed to the algorithm
Save the updated row to database using instance.save()

Since it's easier to scale our task processors than our database replication, I thought I'd shift some of the processing load towards code by doing this:

if original_instance != possibly_updated_instance:
    possibly_updated_instance.save()

Thing is: Django ORM's equality operator only seems to compare primary keys, so we'd incorrectly be dropping changes.

Is there a better way to handle this?

This answer may help https://stackoverflow.com/a/13842223/548562 — Iain Shelvington, Feb 01 '18 at 11:09
Looks interesting. I do still wonder whether it's good practice to offload this responsibility to code though, given https://docs.djangoproject.com/en/2.0/topics/db/optimization/#do-database-work-in-the-database-rather-than-in-python and https://code.djangoproject.com/ticket/27017. — Pieter, Feb 01 '18 at 12:42
The linked answer doesn't have anything to do with DB optimization, It gives you an API for determining if an object has been modified. This API can be used to conditionally run the save command `if instance.has_changed: instance.save()`. Your question is regarding avoiding redundant writes to the DB which this would achieve. As for the Django ticket you linked to - the API also gives you a way of only updating those fields that have changed `if instance.has_changed: instance.save(update_fields=instance.changed_fields)` — Iain Shelvington, Feb 01 '18 at 13:25

Avoiding redundant write operations with Django ORM

0 Answers0