Python threads difference for 3.10 and others

Question

For some, simple thread related code, i.e:

import threading


a = 0
threads = []


def x():
    global a
    for i in range(1_000_000):
        a += 1


for _ in range(10):
    thread = threading.Thread(target=x)
    threads.append(thread)
    thread.start()


for thread in threads:
    thread.join()


print(a)
assert a == 10_000_000

We got different behaviour, based on Python version.

For 3.10, the output is:

❯ python3.10 b.py
10000000

For 3.9, the output is:

❯ python3.9 b.py
2440951
Traceback (most recent call last):
  File "/Users/romka/t/threads-test/b.py", line 24, in <module>
    assert a == 10_000_000
AssertionError

As we do not acquire any lock, for me, results of 3.9 is obvious and expected. Question is why and how 3.10 got "correct" results, while should not?

I'm review changelog for Python 3.10 and there is no anything related to threads or GIL which can bring such results.

I don't know the exact answer, but according to https://stackoverflow.com/questions/1717393/is-the-operator-thread-safe-in-python, the Python interpreter is supposed to only switch active threads every X opcodes. So it could be that they changed X or the `+=` statement uses a different number of opcodes, which accidentally happens to make it threadsafe. — Tin Tvrtković, Nov 17 '21 at 11:23
Fun test: Changing `a += 1` to `a += int(1)` will cause the expected lower number again. — Kelly Bundy, Dec 06 '21 at 14:02

score 24 · Accepted Answer · answered Nov 17 '21 at 14:58

An answer from a core developer:

Unintended consequence of Mark Shannon's change that refactors fast opcode dispatching: https://github.com/python/cpython/commit/4958f5d69dd2bf86866c43491caf72f774ddec97 -- the INPLACE_ADD opcode no longer uses the "slow" dispatch path that checks for interrupts and such.

Python threads difference for 3.10 and others

1 Answers1

Linked

Related