concurrent requests to a async method

Question

I've a simple FastAPI server

@app.get("/")
async def root():
    time.sleep(0.5) # any I/O bound task
    return {"message": "home"}

I'm sending 10 concurrent requests to the server, which would take little over 0.5 seconds if it was a def method instead of async def:

# sending concurrent requests
with concurrent.futures.ThreadPoolExecutor() as executor:
    results = executor.map(send_request, [...])

I understand that I should be using some asynchronous method to sleep on the server but why does my CONCURRENT code behave sequentially and taking 5+ seconds when I'm sending concurrent requests?

Technically there's some latency in every line so does it mean I need to make every line asynchronous?

`async` doesn't run functions at the same time but it switch from one function to another when it execute some function which uses `await` - like `await asyncio.sleep(0.5)` — furas, May 17 '22 at 14:35
You send CONCURRENT requests but server (function `root()`) doesn't run any concurent code so it can't answer for requests in concurrent way - so every request has to wait for end of previous `root()` — furas, May 17 '22 at 14:40
`async` doesn't automagically make your code run in parallell - it's a way for signaling (using `await`) that you're giving up your priority for other tasks to run at the same time because you're going to wait for something that doesn't require CPU time (usually). When you don't include `async` in your function definition, FastAPI creates a ThreadPool to run your functions instead of using async/await, so you get parallel-ish behaviour instead. — MatsLindh, May 17 '22 at 17:28

score 0 · Answer 1 · answered May 24 '22 at 21:55

Defination : asyncio is a library to write concurrent code using the async/await syntax. asyncio is used as a foundation for multiple Python asynchronous frameworks that provide high-performance network and web-servers, database connection libraries, distributed task queues, etc.

asyncio is often a perfect fit for IO-bound and high-level structured network code.

You can see async performance in IO/Bound like : hitting to DB, Calling API and etc. In your example time.sleep isn't IO/bound, you can use await asyncio.sleep(0.5) to imagine you have IO/bound.

As i said, if you wanna use async performance you should consider IO/bounds, in CPU bound like solving algorithm, encryption or other process depends on CPU/bound , async functions not helping you.

Simply put, whenever our system is waiting for an answer from an external service, instead of our system sitting idle, it goes to another task to get the answer from the external system.

concurrent requests to a async method

1 Answers1