Django: Using an F expression for a text field in an update call

Question

In a django view, I need to append string data to the end of an existing text column in my database. So, for example, say I have a table named "ATable", and it has a field named "aField". I'd like to be able to append a string to the end of "aField" in a race-condition-free way. Initially, I had this:

tableEntry = ATable.objects.get(id=100)
tableEntry.aField += aStringVar
tableEntry.save()

The problem is that if this is being executed concurrently, both can get the same "tableEntry", then they each independently update, and the last one to "save" wins, losing the data appended by the other.

I looked into this a bit and found this, which I hoped would work, using an F expression:

ATable.objects.filter(id=100).update(aField=F('aField') + aStringVar)

The problem here, is I get an SQL error, saying:

operator does not exist: text + unknown
HINT:  No operator matches the given name and argument type(s). You might need to add explicit type casts.

Tried changing to "str(aStringVar)" even though its already a string - no luck.. I found a couple django bug reports complaining about similar issues, but I didn't see a fix or a workaround. Is there some way I can cast aStringVar such that it can be appended to the text of the F expression? BTW - also tried "str(F('aField')) + aStringVar" but that converted the result of the F expression to the string "(DEFAULT: )".

Answer 1

You can use the Concat db function.

from django.db.models import Value
from django.db.models.functions import Concat

ATable.objects.filter(id=100).update(some_field=Concat('some_field', Value('more string')))

In my case, I am adding a suffix for facebook avatars URIs like this:

FACEBOOK_URI = 'graph.facebook.com'
FACEBOOK_LARGE = '?type=large'
# ...
users = User.objects.filter(Q(avatar_uri__icontains=FACEBOOK_URI) & ~Q(avatar_uri__icontains=FACEBOOK_LARGE))
users.update(avatar_uri=Concat('avatar_uri', Value(FACEBOOK_LARGE)))

and I get SQL like this (Django 1.9):

UPDATE `user_user` SET `avatar_uri` = CONCAT(COALESCE(`user_user`.`avatar_uri`, ''), COALESCE('?type=large', ''))
WHERE (`user_user`.`avatar_uri` LIKE '%graph.facebook.com%' AND NOT (`user_user`.`avatar_uri` LIKE '%?type=large%' AND `user_user`.`avatar_uri` IS NOT NULL))

The result is all image URIs were changed from http://graph.facebook.com/<fb user id>/picture to http://graph.facebook.com/<fb user id>/picture?type=large

Answer 2

You can override F object in Django with one simple change:

class CF(F):
    ADD = '||'

Then just use CF in place of F. It will place "||" instead of "+" when generating SQL. For example, the query:

User.objects.filter(pk=100).update(email=CF('username') + '@gmail.com')

will generate the SQL:

UPDATE "auth_user" SET "email" = "auth_user"."username" || '@gmail.com'
WHERE "auth_user"."id" = 100 

Answer 3

You can achieve this functionality with Django's select_for_update() operator: https://docs.djangoproject.com/en/dev/ref/models/querysets/#select-for-update

Something like this:

obj = ATable.objects.select_for_update().get(id=100)
obj.aField = obj.aField + aStringVar
obj.save()

The table row will be locked when you call .select_for_update().get(), and the lock will be released when you call .save(), allowing you to perform the operation atomically.

Answer 4

And if you get this running, it isn't thread safe. While your update is running, some other process can update a model not knowing the data in the database is updated.

You have too acquire a lock, but don't forget this senario:

1. Django: m = Model.objects.all()[10]
2. Django: m.field = field
3. Django: a progress which takes a while (time.sleep(100))
4. DB: Lock table
5. DB: Update field
6. DD: Unlock table
7. Django: the slow process is finished
8. Django: m.save()

Now the field update became undone by the model instance in Django (Ghost write)

Answer 5

seems you can't do this. however, what you are trying to do could be solved using transactions

(looks like you are using postgres, so if you want to do it in one query and use raw sql as suggested, || is the concatenation operator you want)