How to avoid cached data in multithread application C++

Question

Can someone clarify the next moment. I saw some implementations of std::queue for multithreading purposes, where all operations of pushing/poping/erasing elements where protected with mutex, but when I see that I imagine next scenario: we have two threads (thread1 and thread2) and they are running on different cores of the processor, thus they have different L1 caches. And we have the next queue:

struct Message {
    char* buf;
    size_t size;
};
std::queue<Message> messageQueue;

Thread1 adds some element to queue, then thread2 tries to access an element with front() method, but what if that piece of memory was previosly cached for this core of the processor (so the size variable may not indicate current size of buf variable, or buf pointer may hold wrong (not updated) address)? I have such problem while designing client/server application on the server side. In my app server is running in one thread, it works directly with sockets, and when it receives new message, it allocates memory for that message and adds this message to some message queue, then other thread accesses this queue, processes message and then deletes it. I am always afraid of caching problems, and because of that I have created my own implementation of queue with volatile pointers. What is the proper way to work with such things? Do I have to avoid using std::list, std::queue with locks? If this problem is impossible, could you please explain why?

The "proper way to work with such things" is to use a mutex, which will correctly handle all sequencing. `volatile` objects accomplish nothing useful, whatsoever, here. The only thing that `volatile` means is that any reference to it is not going to be optimized way. It offers no guarantees that any changes to the object are immediately synchronized to all other execution threads. — Sam Varshavchik, Mar 28 '21 at 18:48
I do believe volatile also prevents caching (variables are read from memory every time they are used). But I agree that is not enough for proper synchronization https://stackoverflow.com/questions/18695120/volatile-and-cache-behaviour. — DeltA, Mar 28 '21 at 19:15
But if i lock some mutex, how can i be shure, that this memory isn't cached, or mutex locking somehow guarantee reading directly from the memory? — Vlad Tishenko, Mar 28 '21 at 20:21
The C++ standard requires that a read operation properly synchronized to occur after a modification operation observes the value written by that modification operation (or possibly some other operation that occurs in between). How this is achieved is an internal implementation detail. CPUs offer machine instructions (fences, memory barriers) that allow the compiler to provide an appropriate implementation. — Igor Tandetnik, Mar 28 '21 at 20:56
Using mutexes is infinitely faster than `volaltile` bugs that you have to spend weeks debugging. — Mooing Duck, Mar 30 '21 at 20:21

score 1 · Accepted Answer · answered Mar 29 '21 at 20:45

It's not your problem. You're writing C++ code. It's the compiler's job to ensure your code makes the CPU and its caches do the right thing, not yours. You just have to comply with the rules for whatever threading standard you are using.

But if i lock some mutex, how can i be shure, that this memory isn't cached, or mutex locking somehow guarantee reading directly from the memory?

You can't. And that's a good thing. Caching massively improves performance and main memory is terribly slow. Fortunately, no modern CPU that you're likely to write multi-threaded code on requires you to sacrifice performance like that. They have incredibly sophisticated optimizations such as cache coherency hardware and prefetch pinning to avoid things that hurt performance that much. What you want is for your code to work, not for it to work awfully.

score 0 · Answer 2 · answered Mar 29 '21 at 20:42

You can safely use mutex locking as long as you use it everywhere you MIGHT access / update in a threaded fashion. This means for this example that adding to / removing from / parsing your list is mutex-guarded.

You also have to be careful that the objects you're storing in your queue are properly protected.

Also, please please please do not use char *. That's what std::string is for.

How to avoid cached data in multithread application C++

2 Answers2