asyncio running from a synchronous (standard) function

Question

Having read the documents and watched a number of videos, i am testing asyncio as an alternative to threading.

The docs are here: https://docs.python.org/3/library/asyncio.html

I have constructed the following code with the expectation that it would produce the following.

before the sleep
hello
world

But in fact is produces this (world comes before hello):

before the sleep
world
hello

Here is the code:

import asyncio
import time


def main():
    ''' main entry point for the program '''
    # create the event loop and add to the loop
    # or run directly.

    asyncio.run(main_async())
    return

async def main_async():
    ''' the main async function '''
    await foo()
    await bar()
    return

async def foo():
    print('before the sleep')
    await asyncio.sleep(2)
    # time.sleep(0)
    print('world')
    return

async def bar():
    print('hello')
    await asyncio.sleep(0)
    return



if __name__=='__main__':
    ''' This is executed when run from the command line '''
    main()

The main() function calls the async main_async() function which in turn calls both the foo and bar async functions and both of those run the await asyncio.sleep(x) command.

So my question is: why is the hello world comming in the wrong (unexpected) order given that i was expecting world to be printed approximately 2 seconds after hello ?

I highly recommend you read [This answer][1], it explains how asyncio works and it might give you a good overview on the subject [1]: https://stackoverflow.com/questions/49005651/how-does-asyncio-actually-work — TheZadok42, Jun 30 '21 at 18:40
This is very interesting (with regards to generators in previous versions of python) and i have read & watched similar. But am still left unaware of what i have done wrong or misunderstood. — D.L, Jun 30 '21 at 18:49

ShadowRanger · Accepted Answer · 2021-06-30T18:57:18.767

3

You awaited foo() immediately, so bar() was never scheduled until foo() had run to completion; the execution of main_async will never do things after an await until the await has completed. If you want to schedule them both and let them interleave, replace:

await foo()
await bar()

with something like:

await asyncio.gather(foo(), bar())

which will convert both awaitables to tasks, scheduling both on the running asyncio event loop, then wait for both tasks to run to completion. With both scheduled at once, when one blocks on an await (and only await-based blocks, because only await yields control back to the event loop), the other will be allowed to run (and control can only return to the other task when the now running task awaits or finishes).

Basically, you have to remember that asyncio is cooperative multitasking. If you're only executing one task, and that task performs an await, there is nothing else to schedule, so nothing else runs until that await completes. If you block by any means other than an await, you still hold the event loop, and nothing else will get a chance to run, even if it's ready to go. So to gain any benefit from asyncio you need to be careful to:

Ensure other tasks are launched in time to occupy the event loop while the original task(s) are blocking on await.
Ensure you only block via await, so you don't monopolize the event loop unnecessarily.

edited Jun 30 '21 at 18:57

answered Jun 30 '21 at 18:48

ShadowRanger

143,180
12
188
271

So are you saying that the error was that the code failed to use the `gather()` function and that two consecutive `awaits` are insufficient (synchronous) ? – D.L Jun 30 '21 at 18:51
I do confirm that *this correction has worked* and am waiting to see if there are other comments otherwise will mark as the accepted answer. – D.L Jun 30 '21 at 18:54
1

@D.L: Yeah, that's basically it. In this case, it's really no different than threading; your original code is similar to running `t = threading.Thread(target=foo); t.start(); t.join()` first, then following up with `t = threading.Thread(target=bar); t.start(); t.join()`, so the `foo` thread is launched and waited for before launching the `bar` thread at all. `asyncio.gather` makes it behave more like `t1 = threading.Thread(target=foo); t1.start(); t2 = threading.Thread(target=bar); t2.start(); t1.join(); t2.join()`, starting both threads before it tries to wait for either so they can both run. – ShadowRanger Jun 30 '21 at 19:01
May i ask, given the explanation above is there a preference for one over the other with respect to `threading` or `asyncio` (for example an efficiency gain) ? – D.L Jul 01 '21 at 01:16
@D.L: Nothing meaningful, at least on CPython. On GIL-free Python interpreters (there aren't many, and they're typically well behind the state of the art), `threading` can provide a major performance boost for CPU bound work (and even on CPython, if it's heavy number crunching with third party packages like `numpy` that release the GIL internally, threading can boost CPU performance). But otherwise, it's pretty similar, and the overhead for `asyncio` is, at least in theory, lower. `asyncio` also makes synchronization easier (if you don't `await`, you're guaranteed not to race). – ShadowRanger Jul 01 '21 at 01:58
1

The downside to `asyncio` being that you have to be more disciplined about explicitly handing control back to the event loop, where `threading`, being preemptive multitasking, hands off between threads automatically. Of course, that makes synchronization more critical/necessary in threaded programs, and opens you to all sorts of race conditions `asyncio` can't have (or can easily avoid). – ShadowRanger Jul 01 '21 at 01:59

asyncio running from a synchronous (standard) function

1 Answers1