I've a "static 64-bit integer variable" that gets updated by only one thread.
All other threads only read from it.
Should i protect this variable using atomic operation (ex. "__sync_add_and_fetch") for safety reasons?
Or is it OK to read(resp. write) from (resp. to) it directly?
I'm still confused because i didn't find a clear answer. I don't know if 've to protect it:
GCC's __sync builtins are obsoleted by its __atomic builtins.
Both reader and writer should be using __atomic operations, like __atomic_load_n and __atomic_store_n (you don't need or want an expensive atomic RMW since there's only one writer.) With __ATOMIC_RELAXED, load and store are as cheap as plain operations (which can't get optimized away into registers.) Normal accesses (not via __atomic builtins) to a plain non-volatile non-_Atomic are never ok with concurrent reads + writes, and will break in practice, e.g. hoisting a load out of a spin-wait loop.
But in new code, you should normally use C11 stdatomic.h with _Atomic types (https://en.cppreference.com/w/c/thread), and the macros available for testing if a certain atomic type is lock-free on the target you're compiling for.
(With the old __sync builtins, I think the design intent was that you'd use volatile for pure-load and pure-store, since they didn't provide __sync builtins for that. Hand-rolling your own atomics with volatile works on normal ISAs with normal compilers like GCC and clang, and the Linux kernel depends on it, but it's not recommended when you can get the job done with C11 stdatomic.h.)
On systems where #if ATOMIC_LLONG_LOCK_FREE > 0 is true, use relaxed (or acq/rel) atomics, depending on what ordering guarantees you need (like if the writer is using the counter to "publish" other data to readers, e.g. if the counter is used as an index into a non-atomic array).
#include <stdatomic.h>
#include <stdint.h>
#if ATOMIC_LLONG_LOCK_FREE > 0
static atomic_uint_fast64_t counter;
// make sure this can inline into readers
uint64_t read_counter() {
return atomic_load_explicit(&counter, memory_order_relaxed); // or m_o_acquire
}
void increment_counter_single_writer() { // one thread only
uint64_t tmp = atomic_load_explicit(&counter, memory_order_relaxed);
// or keep a local copy of the counter in a register and *just* store.
// other threads just see the values we store, doesn't matter how we get them
atomic_store_explicit(&counter, tmp+1, memory_order_relaxed); // or m_o_release
}
// with multiple writers, use atomic_fetch_add_explicit
#else
static uint64_t counter;
static _Atomic unsigned seq;
uint64_t read_counter() { ... }
#endif
Note that some 32-bit systems can do lock-free 64-bit atomics, such as x86 (since P5 Pentium) and some ARM32. See how this compiles on Godbolt with clang for x86 and ARM Cortex-A8 (to pick a random ARM that's not recent).
Otherwise, without lock-free 64-bit atomics, use a SeqLock if the counter doesn't increment too often. (See Implementing 64 bit atomic counter with 32 bit atomics). This still lets the readers be truly read-only so they don't contend with each other for cache lines, they only have to retry if they tried to read while the writer was in the middle of an update. (After the writer is done, the cache line containing the counter can be in Shared state on all cores running reader threads, so they can all get hits in cache.)
For a monotonic counter, the halves of the counter itself can work as a sequence number to detect tearing. Like read the low half before and after reading the high half, retry if different.
A readers/writers lock would force readers to contend with each other to modify the cache line holding the lock, so the total throughput doesn't scale with number of readers. If you have a very-frequently modified counter (so a seqlock would often be in an inconsistent state), you might consider something more clever, like a queue of recent values so readers could check the most recent consistent value or something?
BTW, ATOMIC_LLONG_LOCK_FREE > 0 seems appropriate: ATOMIC_LLONG_LOCK_FREE == 1 means "sometimes lock-free", but in practice there aren't implementations where some objects are lock-free and some aren't. And if there were, we'd hope that a compiler could arrange for a loose global/static variable to be aligned such that it's atomic.
Usually, the atomicity of reading and writing a storage location is the same.
That is to say, if a location cannot be written atomically, it also cannot be read atomically and vice versa.
If a special atomic write is required, it is meaningless to use it unless the reads are also atomic.
For instance, imagine that the 64 bit location is being read using an ordinary read which requires, say, two 32 bit accesses. Suppose the write comes between those two accesses. The read will fetch the second 32 bits from the new value and combine it with the outdated first 32 bits. The only way the write cannot come between the two halves of the read is if the read is atomic. The atomic read knows how to properly interact with atomic write to prevent this situation.
You might be confused by "exceptions" to this rule. In some systems you might see an atomic update such as an increment, mixed with ordinary reads. This is predicated on the assumption that reads and writes are in fact atomic; the special atomic increment is only used to make read/modify/write cycle appear indivisible from the perspective of concurrent writers: if N writers perform this increment at around the same time, it ensures that the location is incremented by N.
Sometimes you might also see correct optimizations whereby an ordinary read is used, even though the underlying data type isn't atomically accessed. In such a situation, the algorithm doesn't care that it reads a "half baked" value.
For instance, in order to simply monitor a memory location to detect a change, you do not require an atomic read. Detecting a change doesn't require retrieval of the correct value. For instance of 0x00000000 is updated to 0x00010001, but the non-atomic read observes the intermediate value 0x00010000, that is still good enough for detecting that the location has changed.
If you must ensure that readers never see a half-baked value, then use atomic reads and writes.
There are other issues, like ordering. Suppose that a writer updates two locations, A and B. In some computing systems, it is possible for a reader to observe the update of B before A. Special "barrier" or "fence" instructions have to be used in addition to any atomic update instructions.
In a higher-level language, the API for these barriers may be built into the semantics of some atomic operations, so you may end up using the atomic instructions just for the sake of these barriers, even if the datum is otherwise atomic.
As a simple answer, I would advise you to protect it (no its not okay to write directly) only when you write to it.
Though you know that if you are doing something like that and threads are not synced you might get different results everytime.
PS. i wanted this to be a comment but my reps too low
Yes. You need a reader-writer lock. That's exactly what they do. Blocks while writing and other than that readers can read at their pleasure
If you're using boost I believe it's boost::shared_mutex
C++ doesn't currently support reader-writer locks though you could implement them yourself.
For more information on reader-writer locks look here
For 64bit target architecture, which has native instructions for read/write 64-bit value as whole, you need no protection like mutexes or seq locks. But some precautions should be done for force compiler to use 64-bit access instructions and for avoid deferring memory accesses when write or read the variable. If the code is compiled with C11 standard or later, atomics (with the relaxed memory order) will be the best choice: they provide all above guarantees. For older compilers volatile specifier for the variable could provide those guarantees in most cases.
Otherwise, for 32-bit architectures, you should use some sort of critical section around write in the updater thread and around reads in the reader threads. It may be a mutex, a read-write lock, a seq-lock, etc.
(There are also 32-bit architectures which support special instructions for 64-bit access. For those targets the solution could be architecture-specific.)
