I'm building an application in which users' information are frequently updated from external APIs. For that, I'm using django-rq
.
The first job is scheduled once a day to get users who need to update their profiles.
With each result returned by the first job, schedule another job to get new user information from remote API and update the user in my database.
# tasks.py import requests from django_rq import job @job('default') def get_users_to_update(): """Get a list of users to who need update""" users = User.objects.filter(some_condition_here...) return users @job('default') def remote_update_user(user): """Calls external API to get new user information""" url = 'http://somwhere.io/api/users/{}'.format(user.id) headers= {'Authorization': "Bearer {}".format(user.access_token)} # Send the request, probably takes long time r = requests.get(url, headers=headers) data = r.json() # new user data for key, value in data.items(): setattr(user, key, value) user.save() return user
I can schedule those two jobs like following:
# update_info.py
import django_rq
scheduler = django_rq.get_scheduler('default')
scheduler.schedule(
scheduled_time=datetime.utcnow(),
func=tasks.get_users_to_update,
interval=24 * 60 * 60
)
scheduler.schedule(
scheduled_time=datetime.utcnow(),
func=tasks.remote_update_user,
interval=24 * 60 * 60
)
However, this is certainly not what I want. I'm wondering if there is any way in django-rq
to notify when get_users_to_update
is finished, get its results and schedule the remote_update_user
.
rq
allows depends_on
for declaring the dependent job a submitted job, but it seems that such functionality isn't available in django-rq
.