0

I am doing some image processing, specifically making thumbnails. After thumbnails are complete, I update a field in my database image table:

Image.objects.filter(file_name__in=images).update(thumbnailed=True)

The table has gotten to be a medium size (about 2 million records) and I started getting the following error:

Lock wait timeout exceeded; try restarting transaction

In looking for a solution, I came across this thread:

Getting "Lock wait timeout exceeded; try restarting transaction" even though I'm not using a transaction

It has some good ideas including increasing the timeout config setting.

Is there a way to more efficiently run this update from Django that would not require changes to the database configuration?

Community
  • 1
  • 1
bcollins
  • 3,379
  • 4
  • 19
  • 35
  • 2
    How large is images? Because IN statements can take quite a while to evaluate. Is there an index on file_name? Also if possible it's better to do a check on the id, since id checks are generally cheaper then varchar comparisons. – EWit Oct 07 '13 at 21:43
  • Ahh thanks for the point regarding the ids. The images table is around 2 million records. I am going to switch the query to use the id field and see if that helps. – bcollins Oct 07 '13 at 22:00
  • How large is the images list which you pass to file_name__in was what I meant. – EWit Oct 07 '13 at 22:29
  • Is your db in the same box as the Django process? If `images` list is very large, it might be taking a while just to transfer the SQL query to the db server. Either way, using ids should make things faster. – miki725 Oct 07 '13 at 23:15
  • So the images list was 200 images. I reduced the list size to 50 and the problem subsided for now – bcollins Oct 08 '13 at 18:47
  • Note that `IN` (the SQL clause) also has a limit so keeping your queries in check is a good idea in general. – Burhan Khalid Dec 10 '13 at 07:08

0 Answers0