Update database fields hourly with Python/Django

Question

Suppose I have 1000 user_ids in a table and I would run every hour to get from Google API info and update 3 fields in that table. How would the impact be and how can it be done efficiently?

I've seen this variant:

m = Module.objects.get(user_id=1).update(field_one=100, field_two=200, field_three=300)

And this one:

m = Module.objects.get(user_id=1)
m.field_one = 100
m.field_two = 200
m.field_three = 300
m.save()

Also how can it be done so that it will run every hour and grab that information? Never done something like this.

Why every hour? Your 1000 users won't be on your site 24 hours a day 365 days a year. Most sites have many users, but only a few regular users. The rest lost interest. So you might think of a rule that checks less often. Kickoff a task when a user logs in? — allcaps, Jul 09 '16 at 21:27

score 2 · Answer 1 · edited May 23 '17 at 12:00

Use Redis, Celery to setup asynchronous task queue every hour. Look here https://realpython.com/blog/python/asynchronous-tasks-with-django-and-celery/ for more info on how to setup asych task queue system for django.

Here is the code for tasks.py

from celery.task import periodic_task
from celery.schedules import crontab  


@periodic_task(run_every=crontab(minute=0, hour='*/1'))
def get_data_from_google_api():
    data_from_google =ping_google_api() # ping google api to get data
    return Module.objects.get(user_id=1).update(field_one= data_from_google['field_one'], field_two= data_from_google['field_two'], field_three= data_from_google['field_three'])

Look here for more info :

score 1 · Answer 2 · answered Jul 09 '16 at 20:58

Fof this purpose you need to run background queries with periodic taks.
Here is most popular in django task-queue-libs
For example, if you decide use celery, you can write simple periodic task:

from celery.schedules import crontab
from celery.task import periodic_task

@periodic_task(
name='UPDATE_USER',
run_every=crontab(
    minute='1',
    hour='1,4,7,10,13,16,19,22'))
def update_user():
    #get some value from api
    Module.objects.filter(user_id=1).update(
        field_one=value, field_two=value, field_three=value)

All settings for django you can look in celery docs

Update database fields hourly with Python/Django

2 Answers2