What happens when a variable is read and AT THE SAME TIME another thread MODIFIES IT?

Question

If the reading happens BEFORE or AFTER the update the behavior is obvious, it is not difficult to see, but what if it happens at the same time? Assuming there is nothing else (no synchronization, just a simple variable read and assignment).

from threading import Thread
import threading

value = 0


def thread1():
    global value
    value = 5


def thread2():
    global value
    # What if I want to read this variable and AT THE SAME TIME another
    # thread wants to update it?
    print(value)

    # What if I want to update this variable and AT THE SAME TIME another
    # thread wants to update it too?
    # value = 10


if __name__ == "__main__":
    thread1 = Thread(target=thread1)
    thread2 = Thread(target=thread2)
    thread1.start()
    thread2.start()

The GIL prevents the situation you're asking about. If that's not an adequate explanation, then your question is "what is the GIL", in which case see linked dupe. :) — Samwise, Nov 01 '22 at 23:12
hi @Samwise, great, glad you guided me, I'm new to threads, so it needs a little push. — , Nov 01 '22 at 23:21
Re-opened because, IMO, there is a deeper question here to which "...because of the GIL..." is not an answer. It's a question that can be asked about languages that do not have a GIL. — Solomon Slow, Nov 02 '22 at 00:14

score 0 · Answer 1 · answered Nov 02 '22 at 00:08

0

Computer hardware and computer languages are intentionally designed so that we never have to answer the question "what if X and Y happen at the same time?"* In order to be able to make sense of any computer program—in order to predict how it will behave—we have to be able to assume that the outcome of the program will be the same as if all of the memory writes and all of the memory reads performed by the program's threads happened one-by-one in some order.

Which of the astronomical number of possible orderings are allowed and which are not allowed is defined by a consistency model. There are several well known models that may be enforced by the hardware and software, but understanding how they are different from each other and which ones are appropriate in which situations is a deeper subject than I am prepared to talk about.

My point is though, no single reads or writes to a computer's memory ever happen "at the same time." reading or writing a Python variable practically always is a single, primitive memory operation. In any language, within any single process, If one thread performs a primitive memory operation X and another thread performs a primitive memory operation Y, then the end result will either be the same as if X happened before Y or, as if Y happened before X. There is no other possibility.

* When we're talking about sequences of operations, that's a whole different story. We sometimes say things like, "method calls A and B happened at the same time," but the formal way to say that is to say that the method calls were overlapped. Two sequences of operations are overlapped if there is any moment in time when both of them have started, but neither one of them has finished.

answered Nov 02 '22 at 00:08

Solomon Slow

25,130
5
37
57

Outside of Python, I think it is theoretically possible for e.g. a C thread running on Core A and another C thread running on Core B to both write to the same memory address at *exactly* the same time (since they are running asynchronously on separate circuits). In that scenario, it would be up to the cache subsystem and memory controller what actually happens, but the C language itself would make no guarantees, other than to say "don't do that, so you won't have to find out" – Jeremy Friesner Nov 02 '22 at 00:14
@JeremyFriesner, Re, "it would be up to the cache subsystem and memory controller what actually happens." Yes. And if the computer system is going to be at all reliable, the end result, as seen by any thread, must be the same as if one of those writes happened before the other. If any thread subsequently examines the variable (that's a _third_ memory operation, so there must be at least three possible outcomes) it must either see the value written by Core A, or the value written by Core B, or it must see the value that was there before either of the writes happened. – Solomon Slow Nov 02 '22 at 00:19
Of course, in C and C++, a simple assignment `a=b` can copy more data than what can be fetched or written in a single memory operation. That's a complication that Python programmers don't have to worry about. – Solomon Slow Nov 02 '22 at 00:23
hi @Jeremy Friesner "but the C language itself would make no guarantees, other than to say "don't do that, so you won't have to find out" that is mentioned here, isn't it? http://eel.is/c++draft/basic#intro.races-2 – Nov 02 '22 at 01:04
hi @Solomon Slow, "My point is though, no single reads or writes to a computer's memory ever happen "at the same time." - Yes, I read it right here on stackoverflow AND IT HAS LOGIC, but, then, why do they recommend locking when a variable can be updated by several threads? – Nov 02 '22 at 01:11
"we have to be able to assume that the outcome of the program will be the same as if all of the memory writes and all of the memory reads performed by the program's threads happened one-by-one in some order." C and C++ programmers would beg to differ. Their memory models don't guarantee sequential consistency, and with relaxed atomics, it is entirely possible to get results that are not consistent with any total ordering of the reads and writes, even without data races and UB. Somehow they still manage to make sense of their programs (usually). – Nate Eldredge Nov 02 '22 at 01:13
hi @Nate Eldredge in c++ it is recommended to use some kind of lock for the case of my question, right? – Nov 02 '22 at 01:25
@NateEldredge, I never said, "sequential." Sequential consistency means that the actions performed by any given thread are guaranteed to take effect in _program order._ I only said that the actions performed by a program's threads must take effect in _some_ order. – Solomon Slow Nov 02 '22 at 01:25
@GeorgeMeijer, Re, "why do they recommend locking..._a_ variable." Of course the classic use-case for mutex locks is to protect a _group_ of variables, allowing one thread to make a sequence of changes such that the intermediate steps cannot be seen by other threads. But on multi-processor systems (most modern desktop, server, and mobile platforms) locking a mutex also locally strengthens the "consistency model" that I mentioned. Rule of thumb: Whatever thread A does before it releases some mutex is guaranteed to become visible to thread B by the time thread B locks the same mutex... – Solomon Slow Nov 02 '22 at 01:34
@SolomonSlow: "I only said that the actions performed by a program's threads must take effect in some order." Okay, but that's not true in C++ either. You can have thread 1 write to `a` and `b`, thread 2 observes the write to `a` then `b`, and thread 3 observes the write to `b` then `a`. See for instance https://stackoverflow.com/questions/27807118/will-two-atomic-writes-to-different-locations-in-different-threads-always-be-see/50679223#50679223. It's quite explicit in the C++ memory model that there is no total order on non-`seq_cst` reads and writes. – Nate Eldredge Nov 02 '22 at 01:35
...Without the mutex, if thread A updates some variable (classic example is a boolean flag that signals some other thread, B, to stop) there often is _no_ guarantee of how long it will take before thread B sees the change. Some systems won't even guarantee that B will _ever_ see the change. – Solomon Slow Nov 02 '22 at 01:36
@NateEldredge, I was being lazy there. You are correct. Different threads can disagree about the order in which events happen. And yet, each individual thread must see the events take effect in some definite order. In particular, what I said to Jeremy Friesner still holds true: If thread A writes value V to a single "word" in memory, and thread B writes W to the same word, then any thread in the program that tries to read that word must either get V or W or the initial value. Every possible outcome must be equivalent to the two writes and the read happening _one-by-one_ in some order. – Solomon Slow Nov 02 '22 at 01:43

What happens when a variable is read and AT THE SAME TIME another thread MODIFIES IT?

1 Answers1