multithreading with std::atomic<>

Question

In the small code example below, I do not understand that, when a thread start to increase the counter the thread will write the counter address in the cache that is shared between core and make the counter lock in the cache until this thread have finish writing in the counter, but if another try to add +1 to this counter between the previous thread read write modify, he will see that the data in the cache is lock and then what ? will the second thread will sleep or just wait until the counter in the cache become unlock ..?

If the thread sleep I do not see the benefit between atomic and mutex for small size data:

// this_thread::yield example
#include <iostream>       // std::cout
#include <thread>         // std::thread, std::this_thread::yield
#include <atomic>         // std::atomic

std::atomic<bool> ready (false);
std::atomic<int> counter (0);

void count1m(int id) {
  while (!ready) {             // wait until main() sets ready...
    std::this_thread::yield();
  }
  for (volatile int i=0; i<10000; ++i) {
        counter +=1;}
}

int main ()
{
  std::thread threads[10];
  std::cout << "race of 10 threads that count to 1 million:\n";
  for (int i=0; i<10; ++i) threads[i]=std::thread(count1m,i);
  ready = true;               // go!
  for (auto& th : threads) th.join();
  std::cout << counter;
}

Pretty sure you don't need `volatile int i=0`. `int i = 0` will be enough. — Fureeish, Jul 03 '19 at 11:19
Its not a real "lock", the core simply synchronize the cache lines ( if cache is used ). And yes, as long it is a cache miss, the core waits until cache is updated. The strategy which is used to synchronize the caches/memory is depending on the cpu architecture and a bit broad to write down here. — Klaus, Jul 03 '19 at 11:24
but when the thread wich do not have the "lock" want to update the counter what happen ? does he sleep or something else ? — Benzait Sofiane, Jul 03 '19 at 12:38
@BenzaitSofiane it stalls. Literally just stalls the core until the cache line can be retrieved. — Mgetz, Jul 03 '19 at 12:44
i think i forgot something because i realy do not understand what is means to say : the thread stall the core ? — Benzait Sofiane, Jul 03 '19 at 13:04

score 1 · Answer 1 · answered Jul 03 '19 at 22:38

A "cache lock" is just a core delaying a response to a MESI invalidate or share request.

It's not a mutex or anything, it's something a single core can do by tweaking the way it uses the existing MESI cache-coherency protocol that makes sure cache never has conflicting values for the same address. Other cores don't have to know about the "lock", they just see a delay in the response to a request to share a line. (Which didn't have any hard deadline or expected response time anyway; it varies depending on contention and how many other cores were also waiting for exclusive access to the line to commit their store or RMW.)

See Can num++ be atomic for 'int num'? for the full details. (Arguably not a duplicate because that one starts from a broader premise and covers lots that isn't relevant here.)

The C++ memory model assumes coherent shared memory, and that's what essentially all multi-core machines have. Always (AFAIK) with some variant of the MESI cache-coherency protocol.

will the second thread will sleep?

No, this is all at the hardware level. The OS doesn't know a core is stalled waiting for a cache line.

It's pretty similar to a regular cache miss, waiting for data to arrive from DRAM instead of from another core.

multithreading with std::atomic<>

1 Answers1