4

I'm building an application in which users' information are frequently updated from external APIs. For that, I'm using django-rq.

  • The first job is scheduled once a day to get users who need to update their profiles.

  • With each result returned by the first job, schedule another job to get new user information from remote API and update the user in my database.

    # tasks.py
    import requests
    from django_rq import job
    
    @job('default')
    def get_users_to_update():
        """Get a list of users to who need update"""
        users = User.objects.filter(some_condition_here...)
        return users
    
    @job('default')
    def remote_update_user(user):
        """Calls external API to get new user information"""
        url = 'http://somwhere.io/api/users/{}'.format(user.id)
        headers= {'Authorization': "Bearer {}".format(user.access_token)}
        # Send the request, probably takes long time
        r = requests.get(url, headers=headers)
    
        data = r.json()  # new user data
        for key, value in data.items():
            setattr(user, key, value)
    
        user.save()
    
        return user
    

I can schedule those two jobs like following:

    # update_info.py
    import django_rq

    scheduler = django_rq.get_scheduler('default')

    scheduler.schedule(
        scheduled_time=datetime.utcnow(),
        func=tasks.get_users_to_update,
        interval=24 * 60 * 60
    )

    scheduler.schedule(
        scheduled_time=datetime.utcnow(),
        func=tasks.remote_update_user,
        interval=24 * 60 * 60
    )

However, this is certainly not what I want. I'm wondering if there is any way in django-rq to notify when get_users_to_update is finished, get its results and schedule the remote_update_user.

rq allows depends_on for declaring the dependent job a submitted job, but it seems that such functionality isn't available in django-rq.

erik
  • 61
  • 5
  • Have you found solution for your problem? – Ivan Oct 15 '21 at 01:52
  • @Ivan I used `remote_update_user.delay(user)` at the end of `get_users_to_update`. This way, after the get function is finished, the `remote_update_user` is dispatched, and run with results from the `get_users_to_update` function. – erik Oct 17 '21 at 15:36

0 Answers0