Asyncio asynchronously run a function

Question

In python, I am trying to run an asynchronous function asynchronously in a thread. My code right now is

import time
import asyncio

async def test():
    print('Started!')
    time.sleep(2)
    print('Ok!')

async def main():
    loop = asyncio.get_event_loop()
    start_time = time.time()
    await asyncio.gather(*(loop.run_in_executor(None, test) for i in range(5)))
    print("--- %s seconds ---" % (time.time() - start_time))

asyncio.run(main())

But this gives me the error RuntimeWarning: Enable tracemalloc to get the object allocation traceback.

I have also tried

await asyncio.gather(*(asyncio.to_thread(test) for i in range(5)))

But that does not work with blockingcode (like the time.sleep I have). It starts only one thread and does the time.sleep one by one. How do I solve this?

Can you explain **why** you have a blocking call (like `time.sleep()`) in the middle of your async function? Is the async function provided by someone else? Explaining the situation will improve your chances of getting meaningful assistance. — user4815162342, Apr 16 '21 at 09:01

score 4 · Answer 1 · answered Apr 16 '21 at 08:56

4

Don't mix sync with async code. Your test function is blocking and adding just the async keyword won't make it asynchronous:

import time
import asyncio


def test():
    print("Started!")
    time.sleep(2)
    print("Ok!")


async def main():
    start_time = time.time()

    await asyncio.gather(*(asyncio.to_thread(test) for i in range(5)))

    print("--- %s seconds ---" % (time.time() - start_time))


if __name__ == "__main__":
    asyncio.run(main())

Test:

$ python test.py
Started!
Started!
Started!
Started!
Started!
Ok!
Ok!
Ok!
Ok!
Ok!
--- 2.0036470890045166 seconds ---

answered Apr 16 '21 at 08:56

HTF

6,632
6
30
49

How does the CPU know implicitly on it's own that `time.sleep` can be awaited, in order to ask the event loop to run something else while the `time.sleep` call finishes its execution? – user2340939 Sep 13 '22 at 16:36
1

The [asyncio.to_thread](https://docs.python.org/3/library/asyncio-task.html#asyncio.to_thread) is a coroutine that supports [cooperative multitasking](https://en.wikipedia.org/wiki/Cooperative_multitasking). It yields control back to the event loop while the blocking function runs asynchronously in a separate thread. – HTF Sep 13 '22 at 20:24
But Python can only execute one Python thread at a time, so even when multiple threads are used, the time it takes for all of them to finish will be 2*5=10s, because each thread needs to wait 2 seconds, and they don't run in parallel. – user2340939 Sep 14 '22 at 09:16
2

That's not true, all standard library function that makes a syscall does release the [GIL](https://docs.python.org/3/glossary.html#term-GIL), including the [time.sleep](https://github.com/python/cpython/blob/main/Modules/timemodule.c#L2155-L2166) function. Additionally, Python interpreter sets the thread [switch interval](https://docs.python.org/3/library/sys.html#sys.setswitchinterval) ([5ms by default](https://github.com/python/cpython/blob/main/Python/ceval_gil.c#L219)) to prevent a Python thread from holding the GIL indefinitely. – HTF Sep 14 '22 at 10:57
Well pointed out! Also, if I understood correctly, it seems `time.sleep` uses sockets on a deeper level, and the OS supports waiting on sockets for incoming or outgoing data; upon receiving data it wakes up, and returns the sockets which received data, or the sockets which are ready for writing. `asyncio` then checks for the future object tied to that socket, and sets it to done, so the function (inside the task's coroutine) can continue execution. Source: https://stackoverflow.com/a/51116910/2340939 – user2340939 Sep 14 '22 at 12:21
@user2340939: You're mixing up `asyncio.sleep` (which allows sleeping while yielding control to the event loop) with `time.sleep` (which is entirely `asyncio` unfriendly). – ShadowRanger Oct 04 '22 at 11:20
Then I guess why I see a parallel execution is because I am using `asyncio.to_thread` on functions which use the `time.sleep` inside, and this effectively makes them as if they would be using `asyncio.sleep`? – user2340939 Oct 04 '22 at 11:42

Asyncio asynchronously run a function

1 Answers1

Linked