Django: Save API data to model

Question

Relative new to Django / python. I have a API call request and i got JSON data from it:

[{'node': 'node01', 'cpu': 0.0193098516002146, 'mem': 92328939520, 'status': 'online'}, {'node': 'node02', 'cpu': 0.0100098516002146, 'mem': 72823919520, 'status': 'online'}, {'node': 'node03', 'cpu': 0.0023498516002146, 'mem': 92328939520, 'status': 'online'}]

If i make i more human-readable:

[{'node': 'node01', 'cpu': 0.0193098516002146, 'mem': 92328939520, 'status': 'online'}, {'node': 'node02', 'cpu': 0.0100098516002146, 'mem': 72823919520, 'status': 'online'}, {'node': 'node03', 'cpu': 0.0023498516002146, 'mem': 92328939520, 'status': 'online'}]

I have the following model:

    class Node(models.Model):
        node = models.CharField(max_length=2000, null=True, blank=True)
        cpu = models.FloatField(null=True, blank=True)
        mem = models.FloatField(null=True, blank=True)
        status = models.CharField(max_length=2000, null=True, blank=True)

Now i want to save the API data into the model, so in this case 3 "records" in the DB.

I tried with the following:

    for node in r:
        a = Node(
            cpu=node['cpu'],
            mem=node['mem'],
            node=node['node'],
            status=node['status'],
            )
        a.save()

r is the API call data. But this saves only the last API data (node03). Also the records have to be updated when i do the API call again (to reflect mem changes etc)

you should use `bulk_create`. Refer [Documentataion](https://docs.djangoproject.com/en/4.2/ref/models/querysets/#bulk-create) — Hemal Patel, Apr 19 '23 at 06:36

score 1 · Answer 1 · answered Apr 19 '23 at 06:40

1

Better to go with bulk_create instead loop. try this:

obj_create_list = []
for node in r:
    obj_create_list.append(
       Node(
            cpu=node['cpu'],
            mem=node['mem'],
            node=node['node'],
            status=node['status'],
            )
    )
if obj_create_list:
   Node.objects.bulk_create(obj_create_list)

answered Apr 19 '23 at 06:40

Hemal Patel

878
6
16

Thanks for the help. This does work and the first time it creates 3 records in DB but the second time i call it, it creates another 3 record, is it possible to update the existing records in the DB ? – Matija Terzić Apr 19 '23 at 06:55
yes what will be the criteria to decide its duplicate? – Hemal Patel Apr 19 '23 at 06:57
Every time i call this i just needs to update (all) the info in the records. – Matija Terzić Apr 19 '23 at 07:00
you need some criteria to update records. like i called api i got `node1` in response. i created the record. and then i called api again and i got `node1` in reponse. then it ll search for `node1` in DB if found then i update it. Your requirement sounds like everytime creating objects and deleting old ones which you're referring as update. – Hemal Patel Apr 19 '23 at 07:06
Ok i undertand. Every time i do this API call i got the same amount of data as in my question, except memory (mem) off course is a different value – Matija Terzić Apr 19 '23 at 07:18
yes so you can do update with considering node names as criteria. Try with `bulk_update`. [Doc](https://docs.djangoproject.com/en/4.2/ref/models/querysets/#bulk-update) – Hemal Patel Apr 19 '23 at 07:26
Thank you for all the info, it works now – Matija Terzić Apr 19 '23 at 12:06

score 0 · Answer 2 · answered Apr 19 '23 at 06:32

Try something below -

Model

class BarList(models.Model):
     api_id = models.CharField(max_length=100, null=False)
     user = models.ForeignKey(User,on_delete=models.CASCADE,null=True,related_name="user")

View

def add_list(request,results):
    if request.method == 'POST':
        Bar = BarList(api_id=results,user=request.user)
        Bar.save()
        return HttpResponseRedirect('/')
    else:
        return render(request, 'API/results.html')

score 0 · Answer 3 · answered Apr 19 '23 at 10:09

I think the issue here is not it is creating 3 records in the DB. Because the code you have shared will create three entries in Database, but the variable a will only keep track of last instance which it has created. But if you check like this:

for node in r:
    a = Node(
        cpu=node['cpu'],
        mem=node['mem'],
        node=node['node'],
        status=node['status'],
        )
    a.save()
    print(a)

It should print the all the instances it has created.

Apart from that, what @HemalPatel suggested is the correct appraoch. You should use bulk_create. Also bulk_create has options for updating instances which are suppose to updated (as explained in this answer).

Still, I would suggest that you keep track of all the changes for a specific Node, and you can always get the latest changes. For example using Window function:

from django.db.models import F, Window
from django.db.models.functions import LastValue

nodes= Node.objects.annotate(
    latest_cpu=Window(
        expression=LastValue('cpu'),
        partition_by=['node']
    )
)

nodes.values('node', 'latest_cpu')

Django: Save API data to model

3 Answers3