Bulk updating a table

Question

I want to update a customer table with a spreadsheet from our accounting system. Unfortunately I can't just clear out the data and reload all of it, because there are a few records in the table that are not in the imported data (don't ask).

For 2000 records this is taking about 5 minutes, and I wondered if there was a better way of doing it.

 for row in data:

        try:

            try:
                customer = models.Retailer.objects.get(shared_id=row['Customer'])
            except models.Retailer.DoesNotExist:
                customer = models.Retailer()

            customer.shared_id = row['Customer']
            customer.name = row['Name 1']
            customer.address01 = row['Street']
            customer.address02 = row['Street 2']
            customer.postcode = row['Postl Code']
            customer.city = row['City']

            customer.save()

        except:
            print formatExceptionInfo("Error with Customer ID: " + str(row['Customer']))

score 2 · Answer 1 · edited May 23 '17 at 11:47

2

Look at my answer here: Django: form that updates X amount of models

The QuerySet has update() method - rest is explained in above link.

edited May 23 '17 at 11:47

Community

1
1

answered Oct 01 '10 at 11:18

bx2

6,356
5
36
40

update sets the _same_ value for _all_ elements in the queryset. I don't think this is what the OP wants. – Manoj Govindan Oct 01 '10 at 11:20
@Manoj: eh - you're right of course - seems to me that I've misunderstand the question.. – bx2 Oct 01 '10 at 11:23
Yep. It's true. Each record will have different data. Thanks anyway though. – alj Oct 04 '10 at 10:42

score 1 · Accepted Answer · answered Oct 01 '10 at 09:07

1

I've had some success using this bulk update snippet: http://djangosnippets.org/snippets/446/

It's a bit outdated, but it worked on django 1.1, so I suppose you can still make it work. If you are looking for a quick way to do a one time bulk insert, this is the quickest (I'm not sure I'd trust it for regular use without seriously testing performance).

answered Oct 01 '10 at 09:07

OmerGertel

2,573
1
19
27

Thanks for this ... I'll have a look. – alj Oct 04 '10 at 10:42

score 1 · Answer 3 · edited Oct 01 '10 at 13:50

1

I've made a terribly crude attempt on a solution for this problem, but it's not finished yet and it doesn`t support working with django orm objects directly - yet.

http://pypi.python.org/pypi/dse/0.1.0

It`s not been properly testet and let me know if you have any suggestions on how to improve it. Using the django orm to do stuff like this is terrible.

Thomas

edited Oct 01 '10 at 13:50

Manoj Govindan

72,339
21
134
141

answered Oct 01 '10 at 11:29

Weholt

1,889
5
22
35

Bulk updating a table

3 Answers3