Interlocked functions and atomicity: still confused

Question

I'm still confused about reading and writing variables atomically. So sorry in advance to those who have tried to help in previous questions.

I have been told today that there is no need for any Interlocked function call when reading and writing a 32bit long value, which blows my previous beliefs out of the window. Indeed, this backs up what MSDN says

Simple reads and writes to properly-aligned 32-bit variables are atomic operations. In other words, you will not end up with only one portion of the variable updated; all bits are updated in an atomic fashion

So I then looked at what atomic_long in VS2017 does; its operator= does an _InterlockedExchange() on the current value with the new value.

1. Doesn't this contradict MSDN; if the read/write is atomic, why is this Interlocked function needed?

   On the same MSDN page, we also have

   Simple reads and writes to properly aligned 64-bit variables are atomic on 64-bit Windows. Reads and writes to 64-bit values are not guaranteed to be atomic on 32-bit Windows. Reads and writes to variables of other sizes are not guaranteed to be atomic on any platform.

   So I then step into atomic_bool's 'operator=; it executes a _InterlockedExchange8 call which MSDN tells me is only available on Windows 8 and above.

   This seems to back up the second MSDN quote.

2. If I am using VS 2005 and targeting Windows XP, how would I ensure an atomic read/write of a boolean variable? Would I have to use a mutex or similar?

3. If a read/write operation is not atomic, that leaves it susceptible to being torn; is that correct?

4. I have read this PAQ that says primitive types are not atomic. Again this contradicts what I have been told in my questions and what MSDN has told me. Who is correct?

Answer 1

Well, you quoted an excerpt of the documentation, but not the sentence just following it:

However, access is not guaranteed to be synchronized. If two threads are reading and writing from the same variable, you cannot determine if one thread will perform its read operation before the other performs its write operation.

That is, while the operations are atomic, ie a thread is guaranteed to write all 4 bytes, before another one is allowed to read it, this doesn't mean that you will be getting the correct values, and the order of execution is unknown.

The Interlocked functions make sure that the operation is carried out in a "transactional" manner, ie it will be atomic and all threads/processors will be getting the last updated values - otherwise this is not guaranteed, because of caching (each CPU/core may maintain a copy in its local cache). The Interlocked functions, as well as all synchronization functions make sure that cache contents are synchronized.

For reading/updating any shared data, yes, you need to employ a mutex. For threads of the same process, a critical section would be preferable, as it's incredibly fast (non-measurable timespans, used them to ensure atomicity in screen-output operations, so i can confirm this).

Answer 2

I have been told today that there is no need for any Interlocked function call when reading and writing a 32bit long value

Because it's not true, at least not that generic.

Only applies to x86, and only if the 32bit value type is aligned, and the variable is actually comitted to memory using a single 32bit wide instruction. Which is commonly the case, but by no means guaranteed, neither the alignment nor the atomic store as the compiler may decide to optimize or can not assume alignment and has therefor to split.

Doesn't this contradict MSDN; if the read/write is atomic, why is this Interlocked function needed?

An atomic_long is not aligned by definition, so an atomic exchange is required for formally correct function.

This function generates a full memory barrier (or fence) to ensure that memory operations are completed in order.

And that is the other relevant part about atomics - they form a memory barrier.

Which means that neither the compiler nor the processor can re-order memory accesses. If you had e.g. two plain int variables which you were writing to independently in the same scope, the compiler would be free to batch or re-order the writes as it likes.

As a side effect, the processor will also attempt to synchronize the caches, so you will see updates made to an atomic variable on another core much faster.

This goes beyond the effect of a plain volatile which would also prevent reordering memory access, but could still spend several microseconds waiting for memory writes to be flushed on another core.

If I am using VS 2005 and targeting Windows XP, how would I ensure an atomic read/write of a boolean variable? Would I have to use a mutex or similar?

Yes. Well, the write is atomic, either way, but you really want the memory barrier if you have been using the boolean to signal that another memory region is now safe to access (e.g. spinlock).

If a read/write operation is not atomic, that leaves it susceptible to being torn; is that correct?

Not for a bool, as it is really just one byte, and I'm not aware of any architecture out there which could tear a variable on the sub-byte level.

For all larger types, yes, as aligment is not guaranteed. And as said before, you don't have any guarantees that the compiler used an atomic instruction, even if it had been "for free".