5

On this page I read this:

Coroutines in the asyncio module are not limited by the Global Interpreter Lock or GIL.

But how is this possible if both the asyncio event loop and the threading threads are running in a single Python process with GIL?

As far as I understand, the impact of the GIL on asyncio will not be as strong as on threading, because in the case of threading, the interpreter will switch to useless operations, such as time.sleep(). Therefore, when working with asyncio, it is recommended to use asyncio.sleep().

I understand that these tools are designed for slightly different things, threading is more often used to execute "legacy" blocking code for IO-Bound operations, and asyncio for non-blocking code.

Meetinger
  • 165
  • 8
  • 1
    I call bull on the way this is presented. I'm going to consult an expert in this stuff separately, but I think the wording is totally disingenuous, if not just flatly false. – roganjosh Apr 01 '23 at 16:13
  • The GIL is there to protect Python's state when you're doing threading. When using asyncio, this form of concurrency is not doing threading, so locking is not needed and the GIL doesn't really come into the picture. If you're mixing threading AND asyncio, then the GIL will be there as usual. – wim Apr 01 '23 at 17:35

2 Answers2

6

The answer to the question

Is asyncio affected by the GIL?

depends on what exactly you mean by "affected".

The Python Docs define the GIL as the

mechanism used by the CPython interpreter to assure that only one thread executes Python bytecode at a time.

Since the asyncio event loop always runs in a single thread, I would argue the topic of the GIL is simply irrelevant most of the time, when you are using asyncio.

But of course you can still employ additional threads, when you work with asyncio. (A popular example is the ThreadPoolExecutor used with run_in_executor.)

And in that case, the limitations of the GIL of course apply.

To be fair to the article you cited, a few sentences later the author qualifies his statement:

The concept of a GIL does not make sense in the context of coroutines as all coroutines in an event loop execute within a single thread.

I would be charitable and interpret this as saying essentially "the GIL does not matter in the context of coroutines". And that is fair enough, keeping in mind the caveat I mentioned above.

Daniil Fajnberg
  • 12,753
  • 2
  • 10
  • 41
5

There are really two kinds of answers to this, depending on whether you take the GIL as the concrete implementation or the conceptual limitation. The answer is No'ish for the former, and Yes'ish for the latter.


No, asyncio concurrency is not bound to the GIL.

The GIL exists to synchronise thread concurrency: When more than one thread is active at once, only the thread that owns the GIL may execute Python code. This means that if another thread needs to run, ownership of the GIL must pass from the current executing thread to the other thread. This is realised via preemptive concurrency, meaning the currently executing thread is forcefully paused regularly.
As such, the GIL is the mechanism handling the switching of control between the currently executing thread and all other threads. This is a very wasteful mechanism that becomes worse the more threads there are.

In contrast, asyncio has its own concurrency synchronisation: async/await provides cooperative concurrency by design. That means the currently executing task can voluntarily yield control to any other waiting task. This allows the asyncio event loop to freely schedule tasks.
As such, the GIL is not involved in switching control between the currently executing task and all other tasks. An async framewok such as asyncio can achieve very efficient task switching, regardless of the number of tasks.


Yes, asyncio is bound by the GIL.

While asyncio can efficiently switch tasks, it cannot run more than one task at any moment. As such, it does not actually avoid the conceptual limitation of the GIL: running more than one operation at once. While asyncio applications avoid the inefficiency of the GIL for many operations, it does not avoid the limitation of the GIL for parallel operations.
Notably, solving this limitation would require solving the exact same challenge as for removing the GIL: providing a safe synchronisation when executing Python code in parallel.

In addition, asyncio still cannot execute blocking code concurrently as tasks but must fallback to threading for these. Thus, asyncio's backend for executing blocking code concurrently is directly bound by the GIL.

MisterMiyagi
  • 44,374
  • 10
  • 104
  • 119
  • "That means the currently executing task can voluntarily yield control to any other waiting task." How, if not via the GIL? – roganjosh Apr 01 '23 at 17:41
  • 2
    @roganjosh Via [the power of async/await](https://stackoverflow.com/questions/49005651/how-does-asyncio-actually-work). The TLDR is that an `asyncio` Task can yield the same way a generator can `yield`. You could see `zip` over multiple generators as a (very) primitive event loop, for example. – MisterMiyagi Apr 01 '23 at 17:53