6

I was reading about the GIL and it never really specified if this includes the main thread or not (i assume so). Reason I ask is because I have a program with threads setup that modify a dictionary. The main thread adds/deletes based on player input while a thread loops the data updating and changing data.

However in some cases a thread may iterate over the dictionary keys where one could delete them. If there is a so called GIL and they are run sequentially, why am I getting dict changed errors? If only one is suppose to run at a time, then technically this should not happen.

Can anyone shed some light on such a thing? Thank you.

Charles
  • 61
  • 1
  • 2
  • 2
    Generally, Python threading only makes sense for I/O-bound threads. If you want consistency accessing data structures in parallel, you need explicit locking. If you need CPU-bound parallelism, usually you have to use something entirely different. – 9000 Jan 20 '11 at 15:25
  • cpu-bound tasks should be run using multiprocessing in lieu of threads. using message-queues as task queues with many workers pulling tasks (or having them pushed) is one such scenario for python parallelism. – JerodG Dec 14 '18 at 15:40

4 Answers4

11

They are running at the same time, they just don't execute at the same time. The iterations might be interleaved. Quote Python:

The mechanism used by the CPython interpreter to assure that only one thread executes Python bytecode at a time.

So two for loops might run at the same time, there will just be no (for example) two del dict[index]'s at the same time.

orlp
  • 112,504
  • 36
  • 218
  • 315
6

The GIL locks at a Python byte-code level, and applies to all threads, even the main thread. If you have one thread modifying a dictionary, and another iterating keys, they will interfere with each other.

"Only one runs at a time" is true, but you have to understand the unit of granularity. In the case of CPython's GIL, the granularity is a bytecode instruction, so execution can switch between threads at any bytecode.

Ned Batchelder
  • 364,293
  • 75
  • 561
  • 662
  • from Python glossary page `GIL The mechanism used by the CPython interpreter to assure that only one thread executes Python bytecode at a time. This simplifies the CPython implementation by making the object model (including critical built-in types such as dict) implicitly safe against concurrent access.` If I am not wrong, this means dict is thread safe? – boh Aug 15 '16 at 06:44
  • 1
    It depends on the operations you perform on the dict. This is not thread-safe: `d[k] += 1`. When in doubt, use your own synchronization. – Ned Batchelder Aug 15 '16 at 11:23
3

The gil prevents two threads from modifying the interpreter state simultaneously. It doesn't provide any thread consistency constraints, or any kind of mutex at all on a granularity smaller than the whole process. If you need to read and modify a dict in two threads, you should be using a mutex

SingleNegationElimination
  • 151,563
  • 33
  • 264
  • 304
1

Python switches threads more often than you seem to think it does. You say "only one" is supposed to run at a time, and technically that's true, but it depends on your definition of "one." Python's atomic operations are very small. For example: adding a single item to a dictionary. Iteration over an entire dictionary can be interrupted.

You should use a lock object from the threading library to isolate your program's atomic operations.

Brian Goldman
  • 716
  • 3
  • 11