How to speed up FastAPI - Uvicorn?

Question

I have a FASTApi application, that I'm trying to deploy via Uvicorn Async task using the below cmd:

uvicorn app.main:app --host='0.0.0.0' --port=8080 --workers=5

My system has below config: 12core, 24threads processor - 32GB RAM - 16GB GPU

I tried to perform load test on one GET & one POST APIs (executing detectron2 and Paddle OCR model predictions). GET takes on avg 3ms and POST takes approx 1.7s-4s. Both my functions/APIs are defined with async def prefix

Irrespective of the number of workers(1-8), GET API runs avg 385RPS while POST API runs on avg 3RPS-10RPS. Like even with 5 workers, for POST my RPS is 3. Can't understand why is this happening.

Please suggest where am I going wrong. How can I increase my RPS/throughput? Should I continue to use Uvicorn for production or switch to anything else (production grade)?

Your help will be highly appreciated. Thanks in advance.

Tried to update number of uvicorn workers from 1 to 8 (can't go beyond 8 - have processing restrictions).

You might find [this answer](https://stackoverflow.com/a/71517830/17865804) helpful — Chris, Jul 08 '23 at 05:13

score 0 · Answer 1 · answered Jul 07 '23 at 18:26

0

May be gunicorn running with uvicorn and taking care of workers fastapi link for gunicorn workers

Apart from that, I would suggest:

Find the bottlenecks in the program, where the performance can be improved by optimising the code like using cProfile .
Use asyncio for parallel computing.

answered Jul 07 '23 at 18:26

Prudhviraj

441
3
12

I did use cprofiler, but it's the ocr and model prediction that takes time. So it's alright. – Kiran Roy Jul 08 '23 at 05:11

How to speed up FastAPI - Uvicorn?

1 Answers1