0

I am building an application in Flask API and React.

The first page of the app presents the user with an upload file form. The user selects a file (700 MB) and click uploads.

Once this is done, the backend:

  • Takes the file, unzip it
  • Run some ML model
  • Returns a JSON containing the right data

When this is over, react gets the JSON and renders a new page.

These three steps takes more than 10 minutes therefore I get an error 500 which I believe is due to the long time request timeout.

I would like to know if there is a way to make timeout=None.

I looked for some answers and they suggest to use Celery. However, I am not sure if this is the right approach for my task.

IonicSolutions
  • 2,559
  • 1
  • 18
  • 31
Prova12
  • 643
  • 1
  • 7
  • 25
  • Does this answer your question? [Making an asynchronous task in Flask](https://stackoverflow.com/questions/31866796/making-an-asynchronous-task-in-flask) – IonicSolutions Jan 21 '20 at 17:54
  • While you could increase the timeout, this is not a robust solution. See https://stackoverflow.com/questions/31866796/making-an-asynchronous-task-in-flask for different ways to accomplish your goal. – IonicSolutions Jan 21 '20 at 17:55
  • Might be better to solve this with some kind of event based on when the model finishes its job. – Maximilian Burszley Jan 21 '20 at 18:09
  • @TheIncorrigible1 Thanks, could you give me some details more? – Prova12 Jan 21 '20 at 18:58
  • Goes way beyond the scope of this site to explain "event-driven architecture", but that's the keyword I would suggest looking up. May have luck kicking off the job with a websocket or something of that nature as well. – Maximilian Burszley Jan 21 '20 at 19:11

1 Answers1

0

I second with @TheIncorrigible suggestion to solve with some kind of event driven architecture what you are doing is Web Worker Architecture. Ref

Your problem reminds me one of the AWS service called control tower where launching landing zone of that service takes more than >10min and AWS gracefully handles that. When you try to launch it gives me a banner saying it is progress and would take 1 hour. In console log I noticed they were using Promise(Not exactly sure how they are achieving and how long it can handle).

May be you could try using Promises in react for asynchronous computations. I am not expert but it looks like you can achieve this using that. You may watch this short video for basic understanding.

There is also signalr that allows server code to send asynchronous notifications to client-side web applications. You can check if that can be applied in your case signalr in python dicussion

Shakeel
  • 1,869
  • 15
  • 23