DRF Pagination custom response without model APIView

Question

Hello I have a project in Django, which performs several queries to different databases returning them in several endpoints, for this I use Pandas and DRF (APIViews).

The problem arises when the response is very long and logically the server runs out of memory, I understand that I need to paginate the result but I have not found the way to do it because I do not use models or serializers, I do raw queries with pandas to make the queries.

Is there any way to paginate the results the way I'm doing it?

I leave some snippets of my code.

class AffiliateAPIView(APIView):
    permission_classes = (IsAuthenticated,)  
    def get(self, request):
        insurance = self.request.query_params.get('insurance', None)
        emergensys_df = pd.read_sql_query(general_attention(insurance), engine())
        return Response(emergensys_df.to_dict(orient='records'))

[This](https://stackoverflow.com/q/18107953/6759844) might help — Brian Destura, Oct 07 '21 at 22:17

score 1 · Answer 1 · answered Oct 08 '21 at 16:09

You should implement a streaming response. Django has StreamingHttpResponse for this: https://docs.djangoproject.com/en/3.2/ref/request-response/#streaminghttpresponse-objects Basically, you should do the following:

create a generator, that gets the data from the database by chuncks using LIMIT and OFFSET SQL operators, processes it, and returns by chunks either.
create and return StreamingHttpResponse and pass the generator into it like: return StreamingHttpResponse(your_generator)

DRF Pagination custom response without model APIView

1 Answers1