1

I am working on the SPacy 3.1 NER pipeline. I have created an application using FastAPI and running from the VM server by exposing the IP using the Unicorn server command [uvicorn main: app --host 0.0.0.0 --port 00]. Now, when I call my training API and start the training non of the other API are working, seems like they all are in queue. I have used async and await.

Can anyone guide me on how to run multiple APIs when training is in progress.

Ganesh Motekar
  • 11
  • 1
  • Using async and await doesn't magically make things parallel if the underlying mechanism isn't async (so it needs to be async support all the way in that case). If you can't do that, just drop the `async` part of your FastAPI code and FastAPI will run the code in a threadpool instead, making it work in parallel. – MatsLindh Aug 24 '22 at 08:02
  • @MatsLindh Running the code in a threadpool wouldn't actually make it work in parallel due to CPython's GIL, which ensures that there is never more than one thread of execution for the python interpreter at any given time. To run tasks (especially, CPU-bound tasks) in parallel one could either increase the number of workers or use the standard `multiprocessing` module (I will soon update the linked answer above with more info on that). – Chris Aug 24 '22 at 08:44
  • 1
    @Chris Sure, but in that case it is in regards to single opcodes in the underlying VM that executes the compiled code, not the single statements that a user invokes in their controller. In many cases it's good enough, but sure - multiple workers and multiprocessing is the preferred way. – MatsLindh Aug 24 '22 at 09:05

0 Answers0