Await multiple async functions in python

Question

Awaiting for multiple async functions is not really working asynchronously.

For example, I am expecting the below code that runs multiple await functions to run in ~6 seconds, but it is running like synchronous code and executing in ~10 seconds.

When I tried it in asyncio.gather, it is executing in ~6 seconds.

Can someone explain why is this so?

 # Not working concurrently
 async def async_sleep(n):
    await asyncio.sleep(n+2)
    await asyncio.sleep(n)

start_time = time.time()
asyncio.run(async_sleep(4))
end_time = time.time()
print(end_time-start_time)

# Working concurrently 
async def async_sleep(n):
    await asyncio.gather(
        asyncio.sleep(n+2),
        asyncio.sleep(n)
    )

user4815162342 · Answer 1 · 2019-08-16T17:13:42.430

11

Can someone explain why [gather is faster than consecutive awaits]?

That is by design: await x means "do not proceed with this coroutine until x is complete." If you place two awaits one after the other, they will naturally execute sequentially. If you want parallel execution, you need to create tasks and wait for them to finish, or use asyncio.gather which will do it for you.

edited Aug 16 '19 at 17:13

answered Aug 16 '19 at 10:06

user4815162342

141,790
18
296
355

1

You should consider inserting an example here, instead of pointing to the documentation. e.g. I tried this ```await asyncio.gather(task1, task2)```, which runs the tasks in sequence, rather parallel. I don't think your statement is correct. – Houman Jul 24 '21 at 14:57
2

@Houman `asyncio.gather` is definitely intended to run things in parallel. If you're getting things in sequence, you're probably not awaiting, i.e. you're running sync code. – user4815162342 Jul 24 '21 at 21:19
It is co-operative multitasking. If one wanted to run things - in parallel proper - one would use the `multiprocessing` module. The `asyncio` and `threading` modules run on a single cpu and hence can only run one at a time. The `multiprocessing` module runs processes on different cpus and therefore can run multiple at the same time. – tread Apr 22 '23 at 13:21
1

@tread I'm well aware of asyncio being single-threaded as well as of the GIL. While it's true that Python runs things on a single CPU, it doesn't mean that it can't do things in parallel, especially when things are IO waits. Parallel as used in this (almost 4 years old) answer didn't refer to what you choose to call "proper parallel", but to "not in sequence". The commenter whose `asyncio.gather()` was executing the tasks in series was likely dealing with non-async blocking code and that was his issue, not the GIL or a lack of "proper" parallelism. – user4815162342 Apr 22 '23 at 15:17

tread · Answer 2 · 2023-04-22T20:42:31.863

What does await do?

Suspend the execution of coroutine on an awaitable object. Can only be used inside a coroutine function.

What does asyncio.gather do?

Run awaitable objects in the aws sequence concurrently. (Where aws is an iterable of awaitable objects)
If any awaitable in aws is a coroutine, it is automatically scheduled as a Task.

When one uses the await keyword - it causes the current coroutine to suspend execution at that point and wait for the awaitable to be done.

Coroutines are different from Tasks - but they are both awaitables.

Checking the types of the objects in the case presented:

$ python -m asyncio
>>> import asyncio
>>> type(asyncio.sleep(2))
<class 'coroutine'>
>>> type(asyncio.create_task(asyncio.sleep(2)))
<class '_asyncio.Task'>

Executing a coroutine or task directly with await - will make the caller wait for it to complete. The difference with a task is that on creation of the task with asyncio.create_task the task is scheduled (and executed if the event loop is not busy). So oftentimes when the awaiting happens the task is already complete.

An alternate way to run the tasks without using asyncio.gather is by creating tasks with asyncio.create_task. This will achieve the ~6 second result.

async def async_sleep(n):
    task_sleep_1 = asyncio.create_task(asyncio.sleep(n+2))
    task_sleep_2 = asyncio.create_task(asyncio.sleep(n))
    
    await task_sleep_1
    await task_sleep_2

Clarifications:

There is no parallel at all when talking asyncio. Only context switching.
The tasks will run in sequence - in sequence concurrently / overlapping. Not waiting for a result until moving on (blocking).

Sources:

Await multiple async functions in python

2 Answers2