Making data reads/writes atomic in C11 GCC using ?

Question

I have learned from SO threads here and here, among others, that it is not safe to assume that reads/writes of data in multithreaded applications are atomic at the OS/hardware level, and corruption of data may result. I would like to know the simplest way of making reads and writes of int variables atomic, using the <stdatomic.h> C11 library with the GCC compiler on Linux.

If I currently have an int assignment in a thread: messageBox[i] = 2, how do I make this assignment atomic? Equally for a reading test, like if (messageBox[i] == 2).

Answer 1

For C11 atomics you don't even have to use functions. If your implementation (= compiler) supports atomics you can just add an atomic specifier to a variable declaration and then subsequently all operations on that are atomic:

_Atomic(int) toto = 65;
...
toto += 2;  // is an atomic read-modify-write operation
...
if (toto == 67) // is an atomic read of toto

Atomics have their price (they need much more computing resources) but as long as you use them scarcely they are the perfect tool to synchronize threads.

Answer 2

If I currently have an int assignment in a thread: messageBox[i] = 2, how do I make this assignment atomic? Equally for a reading test, like if (messageBox[i] == 2).

You almost never have to do anything. In almost every case, the data which your threads share (or communicate with) are protected from concurrent access via such things as mutexes, semaphores and the like. The implementation of the base operations ensure the synchronization of memory.

The reason for these atomics is to help you construct safer race conditions in your code. There are a number of hazards with them; including:

ai += 7;

would use an atomic protocol if ai were suitably defined. Trying to decipher race conditions is not aided by obscuring the implementation.

There is also a highly machine dependent portion to them. The line above, for example, could fail [1] on some platforms, but how is that failure communicated back to the program? It is not [2].

Only one operation has the option of dealing with failure; atomic_compare_exchange_(weak|strong). Weak just tries once, and lets the program choose how and whether to retry. Strong retries endlessly. It isn't enough to just try once -- spurious failures due to interrupts can occur -- but endless retries on a non-spurious failure is no good either.

Arguably, for robust programs or widely applicable libraries, the only bit of you should use is atomic_compare_exchange_weak().

[1] Load-linked, store-conditional (ll-sc) is a common means for making atomic transactions on asynchronous bus architectures. The load-linked sets a little flag on a cache line, which will be cleared if any other bus agent attempts to modify that cache line. Store-conditional stores a value iff the little flag is set in the cache, and clears the flag; iff the flag is cleared, Store-conditional signals an error, so an appropriate retry operation can be attempted. From these two operations, you can construct any atomic operation you like on a completely asynchronous bus architecture.

ll-sc can have subtle dependencies on the caching attributes of the location. Permissible cache attributes are platform dependent, as is which operations may be performed between the ll and sc.

If you put an ll-sc operation on a poorly cached access, and blindly retry, your program will lock up. This isn't just speculation; I had to debug one of these on an ARMv7-based "safe" system.

[2]:

#include <stdatomic.h>
int f(atomic_int *x) {
    return (*x)++;
}
f:
        dmb     ish
.L2:
        ldrex   r3, [r0]
        adds    r2, r3, #1
        strex   r1, r2, [r0]
        cmp     r1, #0
        bne     .L2       /* note the retry loop */
        dmb     ish
        mov     r0, r3
        bx      lr

Answer 3

The most portable way is to use one of the C11 atomic variables. You can also use a spinlock atomic operation to guard non-atomic variables. Here is a simple pthread produce/consumer example to play with, modify as desired. Notice that the cnt_non and cnt_vol can be corrupted.

atomic_uint cnt_atomic;
int cnt_non;
volatile int cnt_vol;

typedef atomic_uint lock_t;
lock_t lockholder = 0;
#define LOCK_C 0x01
#define LOCK_P 0x02
int cnt_lock;  /* not atomic on purpose to test spinlock */
atomic_int lock_held_c, lock_held_p;

void
lock(lock_t *bitarrayp, uint32_t desired)
{
    uint32_t expected = 0; /* lock is not held */

    /* the value in expected is updated if it does not match
     * the value in bitarrayp.  If the comparison fails then compare
     * the returned value with the lock bits and update the appropriate
     * counter.
     */
    do {
        if (expected & LOCK_P) lock_held_p++;
        if (expected & LOCK_C) lock_held_c++;
        expected = 0;
    } while(!atomic_compare_exchange_weak(bitarrayp, &expected, desired));
}

void
unlock(lock_t *bitarrayp)
{
    *bitarrayp = 0;
}

void*
fn_c(void *thr_data)
{
    (void)thr_data;
    
    for (int i=0; i<40000; i++) {
        cnt_atomic++;
        cnt_non++;
        cnt_vol++;

        /* lock, increment, unlock */
        lock(&lockholder, LOCK_C);
        cnt_lock++;
        unlock(&lockholder);
    }

    return NULL;
}

void*
fn_p(void *thr_data)
{
    (void)thr_data;
    for (int i=0; i<30000; i++) {
        cnt_atomic++;
        cnt_non++;
        cnt_vol++;

        /* lock, increment, unlock */
        lock(&lockholder, LOCK_P);
        cnt_lock++;
        unlock(&lockholder);
    }
    
    return NULL;
}

void
drv_pc(void)
{
    pthread_t thr[2];

    pthread_create(&thr[0], NULL, fn_c, NULL);
    pthread_create(&thr[1], NULL, fn_p, NULL);

    for(int n = 0; n < 2; ++n)
        pthread_join(thr[n], NULL);
    
    printf("cnt_a=%d, cnt_non=%d cnt_vol=%d\n", cnt_atomic, cnt_non, cnt_vol);
    printf("lock %d held_c=%d held_p=%d\n", cnt_lock, lock_held_c, lock_held_p);

}

Answer 4

that it is not safe to assume that reads/writes of data in multithreaded applications are atomic at the OS/hardware level, and corruption of data may result

Actually non composite operations on types like int are atomic on all reasonable architecture. What you read is simply a hoax.

(An increment is a composite operation: it has a read, a calculation, and a write component. Each component is atomic but the whole composite operation is not.)

But atomicity at the hardware level isn't the issue. The high level language you use simply doesn't support that kind of manipulations on regular types. You need to use atomic types to even have the right to manipulate objects in such a way that the question of atomicity is relevant: when you are potentially modifying an object in use in another thread.

(Or volatile types. But don't use volatile. Use atomics.)