asyncio run from sync function in multiple workers

Question

I am really struggling to understand the interaction between asyncio event loop and multiple workers/threads/processes.

I am using dash: which uses flask internally and gunicorn.

Say I have two functions

def async_download_multiple_files(files):
    # This function uses async just so that it can concurrently send
    # Multiple requests to different webservers and returns data.

def sync_callback_dash(files):
    # This is a sync function that is called from a dash callback to get data
    asyncio.run(async_download_multiple_files(files))

As I understand, asyncio.run runs the async function in an event loop but blocks it: From Python Docs

While a Task is running in the event loop, no other Tasks can run in the same thread.

But what happens when I run a WSGI server like Gunicorn with multiple workers.

Say there are 2 requests coming in simultaneously, presumably there will be multiple calls to sync_callback_dash which will happen in parallel because of multiple Gunicorn workers.

Can both request 1 and request 2 try to execute the asyncio.run in parallel in different threads\processes ? Will one block the other ?

If they can run in parallel, what is the use of having asyncio workers that Gunicorn offers?

Answer 1

I answered this question with the assumption that there is some lack of knowledge on some of the fundamental understandings of threads/processes/async loop. If there was not, forgive me for the amount of detail.

First thing to note is that processes and threads are two separate concepts. This answer might give you some context. To expand:

Processes are run directly by the CPU, and if the CPU has multiple cores, processes can be run in parallel. Inside processes is where threads are run. There is always at least 1 thread per process, but there can be more. If there are more, the process switches between which thread it is executing after every (specific) millisecond (dictated by things out of the scope of this question)- and therefore threads are not run in absolute parallel, but rather constantly switched in and out of the CPU (at least as it pertains to Python, specifically, due to something called the GIL). The async loop runs inside a thread, and switches context relating specifically to I/O-bound instructions (more of this below).

Regarding this question, it's worth noting that Gunicorn workers are processes, and not threads (though you can increase the amount of threads per worker).

The intention of asynchronous code (with the use of async def, await, and asyncio) is to speed-up performance as it specifically relates to I/O bound tasks. Stuff like getting a file from disk, sending/receiving a network request, or anything that requires a physical piece of your computer - whether it is SSD, or the network card - other than the CPU to do some work. It can also be used for large CPU-bound instructions, but this is usually where threads come in. Note that I/O bound instructions are much slower than CPU bound instructions as the electricity inside your computer literally has to travel further distances, as well as perform extra steps in the hardware level (to keep things simple).

These tasks waste the CPU time (or, more specifically, the current process's time) on simply waiting for a reply. Asynchronous code is run with the help of a loop that auto-manages the context switching of I/O bound instructions and normal CPU bound instructions (dependent on the use of await keywords) by leveraging the idea that a function can "yield" control back to the loop, and allow the loop to continue processing other pieces of code while it waits. When async code sends an I/O bound instruction (e.g. grab the latest packet from the network card), instead of sitting still and waiting for a reply it will switch the current process' context to the next task in its list to speed up general execution time (adding that previous I/O bound call to this list to check back in later). There is more to this, but this is the general gist as it relates to your question.

This is what it means when the docs says:

While a Task is running in the event loop, no other Tasks can run in the same thread.

The async loop is not running things in parallel, but rather constantly switching context between different instructions for a more optimized CPU + I/O relationship/execution.

Processes, in the other hand, run in parallel in your CPU assuming you have multiple cores. Gunicorn workers - as mentioned earlier - are processes. When you run multiple async workers with Gunicorn you are effectively running multiple asyncio.loop in multiple (independent, and parallel-running) processes. This should answer your question on:

Can both request 1 and request 2 try to execute the asyncio.run in parallel in different threads\processes ? Will one block the other ?

If there is ever the case that one worker gets stuck on some extremely long I/O bound (or even non-async computation) instruction(s), other workers are there to take care of the next request(s).

Answer 2

With asyncio it is possible to run a separate event loop in each thread. Both will run in parallel (to the extent the Python Interpreter is capable). There are some restrictions. Communication between those loops must use threadsafe methods. Signals and subprocesses can be handled in the main thread only.

Calling asyncio.run in a callback will block until the asyncio part completely finishes. It is not clear from your question if this is what you want.

Alternatively, you could start a long running event loop in one thread and use asyncio.run_coroutine_threadsafe from other threads. Read the docs with an example here.