fix long running filter task in django views

Question

    input_df = pd.read_excel(uploadedfile)
    variable_column = code
    search_type = 'icontains'
    filter = variable_column + '__' + search_type
    
    li = []
    for i in input_df["KT_DB"]:
        qset=KT_DB.objects.filter(**{ filter: i})
        df = pd.DataFrame.from_records(qset.values())
        li.append(df)
    frame = pd.concat(li, axis=0, ignore_index=True)
    with BytesIO() as b:
        # Use the StringIO object as the filehandle.
        writer = pd.ExcelWriter(b, engine='xlsxwriter')
        frame.to_excel(writer, sheet_name = 'Sheet1', index = False)
        writer.save()
        filename = 'KW'
        content_type = 'application/vnd.ms-excel'
        response = HttpResponse(b.getvalue(), content_type=content_type)
        response['Content-Disposition'] = 'attachment; filename="' + filename + '.xlsx"'
        return response

here, filter multiple keywords from the database table, it takes more time to execute. getting request time out before function executes.

score 0 · Accepted Answer · answered Oct 06 '22 at 07:49

You have to make this work in other thread, it is probably better.

Celery With Django for Background Task that means process the excel file with celery task: https://docs.celeryq.dev/en/stable/getting-started/introduction.html You can show different solution on this thread: Django Background Task

OR

As a quick hack, you can use async task with django. Display a "processing" message to user, waiting the file is available...

Never test but this library exists too: https://django-q.readthedocs.io/en/latest/configure.html

fix long running filter task in django views

1 Answers1