How does asyncio (python) work?

Question

As reported in this question(second answer) "yield from coroutine()" is like calling a function. I don't understand how something like yield from, that actually waits until the coroutine is finished should enable concurrency and non blocking operations.

For example in Golang you just reason by the go keyword.

When you do something like:

func function1(){
    go function2()
... Some channel communication here
}

what happens is that ideally the 2 task start in parallel and can communicate concurrently using channels.

This tasks can easily be made parallel(increasing the number of procs) and the logic behind is more straightforward to me.

Please someone help me please to understand this stuff.

'yield from' provides a point at which asyncio can actively hop between tasks so that blocking operations don't impede other tasks from running. Actual parallelism will require use of the ThreadPoolExecutor or ProcessPoolExecutor from the concurrent.futures module, which can be run in asyncio using the run_in_executor() method. — songololo, May 20 '15 at 09:15

mgilson · Accepted Answer · 2015-05-20T16:26:06.310

I think (and to be fair, I'm not an expert on this side of python) that the answer is that coroutines by themselves don't really buy you magical parallelism. A coroutine is a way to say "work on this task until ... and then pause." After it pauses, flow control can go work on something else until you decide to start it up again.

Where you can get multiple things working at the same time is if your coroutine ultimately starts something that can be done asyncronously -- e.g. if you can spin up a thread in the coroutine and the thread is working on something that just so happens to release the GIL (i.e. IO). Then you can start the thread in your co-routine and then pause that execution delegating to other tasks. When you're done with those other tasks, hopefully your IO will be complete and you can go on your merry way without having to wait for the IO at all. I believe that this is what asyncio is doing for you.

^{Note, if you're looking for a good read on co-routines, it's worth checking out this presentation (concurrency starts at about slide #75).}

How does asyncio (python) work?

1 Answers1