0

I'm new to working with ASGI and I'm having some performance issues with my Django API server so any insight would be greatly appreciated.

After running some load tests using JMeter it seems that the application is able to handle under 10 or so simultaneous requests as expected, but any more than that and the application's threads seem to lock up and the response rate vastly diminishes (can be seen in sample #14 and up on the 'Sample Time(ms)' column in the screenshot provided). It's as if requests are only being processed on 1-2 threads after that point as opposed to the configured 4. Here is my startup.sh config:

gunicorn --workers 8 --threads 4 --timeout 60 --access-logfile \
'-' --error-logfile '-' --bind=0.0.0.0:8000  -k uvicorn.workers.UvicornWorker \
--chdir=/home/site/wwwroot GSE_Backend.asgi

I am aware that the individual API responses are quite slow. That is something I want to address. However, is there perhaps something I am doing wrong on the ASGI level here? Thanks.

Edit: I have also added a second load test screenshot. This one uses an API that just sleeps for 10 seconds so should have little CPU/memory impact. A similar pattern occurs again where fewer and fewer requests are responded to at one time.

Server load test using database query apiServer load test using sleep api

Slavi Paskalev
  • 13
  • 4
  • Can you share more details of your API and what it's doing, with code? Python will only execute a single thread at a time so if you have very long running CPU bound requests you will only be able to handle around 1 simultaneous request per worker – Iain Shelvington Dec 08 '21 at 19:22
  • @IainShelvington I'm not allowed to directly share the code here but I can explain the API used in the load test. Basically it just pulls all rows from a table in a database (around 500), serializes the query-set, and returns the result. The max database connections are never exceeded in the test, and adding more vCores to the database does not make a difference here. What is weird is that the first 10 requests are all answered within 7 seconds, but the next 10 take 11 seconds. The 10 after that are even slower. Thank you for the response. – Slavi Paskalev Dec 08 '21 at 20:09
  • I don't think the `--threads` option does anything when you use the uvicorn worker. All the requests are made at the same time and are all waiting for their turn to execute, the timing does not seem that surprising if you only have 8 requests being processed at a time. The size of the responses seems rather large, are you running out of memory? – Iain Shelvington Dec 08 '21 at 20:21
  • @IainShelvington I will try getting rid of the --threads part. Also I have been trying to get Azure to give me a report of the memory and CPU usage but it's throwing errors at me right now. It's something I will look into for sure though. I did run another load test with an API that just sleeps for 10 seconds and thus should have a fairly minimal impact to CPU/memory and a similar pattern occurs where fewer and fewer requests are processed as time goes on. I have made an edit to the original question if you would like to see. Again, thanks for the replies. – Slavi Paskalev Dec 08 '21 at 21:36
  • That seems consistent, you are handling 1 request per worker at a time. The first 8 requests are completed after 10 seconds and then the next batch start and take another 10 seconds, although some requests/workers seem to get lost here. Your view/handler isn't async is it? If you change it to be async and use `await asyncio.sleep(10)` instead of `time.sleep(10)` you should get all the requests finishing within a few seconds of each other hopefully, this should confirm that it's just a scheduling issue rather than something else – Iain Shelvington Dec 08 '21 at 21:47
  • @IainShelvington Yes exactly. Some workers get lost with every other batch and we get less responses every 10 seconds. The views are not async. I know Django now supports async in both wsgi/asgi but we are using Djagno Rest Framework here which I don't think is meant for it? I tried setting a simple view to async to accommodate your suggestion but am running into an `AssertionError: Expected a Response, HttpResponse or HttpStreamingResponse to be returned from the view, but received a ` I do return a Response object but adding 'async' to the view get() causes this. – Slavi Paskalev Dec 08 '21 at 23:47
  • If you're not using asnyc views then you'll most likely be better off using a sync worker class and WSGI, there is a bit of overhead running sync views in an async worker and almost no benefit. I would try the default gunicorn sync worker with threads and see how that performs, there are quite a few to try out – Iain Shelvington Dec 09 '21 at 00:21
  • @IainShelvington Alright thank you for the suggestion. I'll try it out and see how it performs. – Slavi Paskalev Dec 09 '21 at 02:21

0 Answers0