Asynchronous HTTP call inside HTTP cycle of Django Rest Framework

Question

I need to call external service on the get_queryset function of one of my viewsets on views.py, but I don't need to wait for the response of external function.

How can I make the following requests.post line just triggered and passed to next line without waiting for the result?

class SomeViewSet(viewsets.ModelViewSet):
    def get_queryset(self):
        ....
        requests.post(urlOfApi+token)
        .....
        return Some.objects.all()

Try to write [celery task](http://docs.celeryproject.org/en/latest/django/first-steps-with-django.html), and make the external API request through celery — JPG, Sep 13 '18 at 06:56
or you can use [threading](https://stackoverflow.com/questions/2846653/how-to-use-threading-in-python) — JPG, Sep 13 '18 at 07:05

score 2 · Answer 1 · answered Sep 13 '18 at 10:22

I implemented an endpoint to run a background task. When you hit the post the endpoint returns the id of the job. Then you can hit the get endpoint with the job task id and you get back the result or in progress if the job is still running.

I didn't use celery to accomplish this task, but django-q that I found easier than celery and moreover django-q uses all the batteries django has already. Here an example of API View I implemented with django-q.

from django.urls import reverse
from django_q.models import Task, OrmQ
from django_q.tasks import async, result
from rest_framework.exceptions import NotFound
from rest_framework.response import Response
from rest_framework.views import APIView


def the_function_to_run():
    # do something here
    pass


class YourTaskApiView(APIView):

    def get(self):

        # you mustr construct the get url to take the id
        task_id = self.kwargs.get('task_id')

        try:
            task = Task.objects.get(id=task_id)
        except Task.DoesNotExist:
            task = None

        # here the task is on processing
        if task:
            return Response({
                'id': task_id,
                'result': result(task_id),
                'started': task.started,
                'stopped': task.stopped,
                'status': 'DONE' if task.stopped else 'RUNNING',
                'success': task.success,
            })
        else:
            # here you find the task in the query (not even processed, but waiting for the cluster to process it)
            for q in OrmQ.objects.all():
                if q.task_id() == task_id:
                    task = q.task()
                    return Response({
                        'id': task['id'],
                        'started': task['started'],
                        'status': 'WAITING', # or ON QUEUE
                        'stopped': None,
                        'success': None,
                    })

        return NotFound()


    def post(self, request):

        # run the task as async
        task_id = async(the_function_to_run)

        # create the reverse for the get so the client already has the link to query for
        # the task status
        link = reverse('yourapp:yournamespace:yourview', kwargs={'task_id': task_id}, request=request)
        return Response(data={'task_id': task_id, 'link': link})

score 0 · Answer 2 · answered Sep 13 '18 at 08:07

You need to implement some kind of background jobs for this kind of work.

As mentioned in the comment - one way is to use celery.

If you don't want to go through that way. You can try the following:

https://django-background-tasks.readthedocs.io/en/latest/

Perhaps, not as powerful as the celery way, but easier to setup and run.

You simply implement a function to perform the work ( in your case, making http request to another service) , decorate it with background decorator. And then, in your viewset, you simple call the function and it will be scheduled. You can even choose the time when.

Asynchronous HTTP call inside HTTP cycle of Django Rest Framework

2 Answers2