Correct way to ensure sharing with std::hardware_constructive_interference_size

Question

What is the correct and portable way to ensure true sharing in a struct small enough to fit in a cacheline? Is it enough to just ensure that the struct is small enough? Or does it also have to be aligned on the cache boundary?

For example, assuming the size of a cacheline is 64 bytes, is the following enough?

struct A {
  std::uint32_t one;
  std::uint32_t two;
};

Or do I have to do this?

struct alignas(std::hardware_constructive_interference_size) A {
  std::uint32_t one;
  std::uint32_t two;
};

Note: This will always be on the stack, so no over-aligned memory allocations should be required.

Another followup, is this enough to ensure no false-sharing?

struct A {
public:
  alignas(hardware_destructive_interference_size) std::uint32_t one;
  alignas(hardware_constructive_interference_size) std::uint32_t two;
};

or does one have to do this (in the case where say hardware_constructive_interference_size < hardware_destructive_interference_size?)

struct A {
public:
  alignas(hardware_destructive_interference_size) std::uint32_t one;
  alignas(hardware_destructive_interference_size) std::uint32_t two;
};

Looks about right. Without the extra alignment, the two class members could certainly end up on different cache lines. — Kerrek SB, Feb 03 '19 at 01:34
@KerrekSB Could you make that an answer I can accept? Could you also share some resource that points to the fact that cachelines typically need to have memory loaded on boundaries aligned on the cacheline size? — Curious, Feb 03 '19 at 01:36
I'm not sure what your second concern is: note that cache lines are merely a subdivision of the address space into, say, 64-byte chunks. They don't "move around", if you will. So if your data crosses a 64-byte boundary, it occupies (at least) two cache lines. You don't get to choose where the cache line goes. — Kerrek SB, Feb 03 '19 at 01:45
@KerrekSB I guess that is what I wanted clarification on I guess, Also - I added a smaller followup that is somewhat related — Curious, Feb 03 '19 at 01:49

Johannes Overmann · Accepted Answer · 2019-02-03T02:06:44.917

The second variant is currently the best you can do.

However, there is no 100% portable way to align to cache line sizes. The constants hardware_constructive_interference_size and hardware_destructive_interference_size are just hints. They are best guesses of the compiler. Ultimately you do not know the L1 cache line size at compile time.

But in practice this usually does not matter, since for most architectures there is a typical cache line size, like 64 bytes for x86.

Even more, for small structs like in your example, it is always sufficient to naturally align the struct to make sure it is completely within a cache line. In your concrete example this means that

struct alignas(8) A {
  std::uint32_t one;
  std::uint32_t two;
};

will always ensure true sharing, regardless of the actual L1 cache line size at runtime, provided that the cache line size is 8 bytes or bigger. (If it is smaller you will never have true sharing trivially.)

Regarding the follow-up question: The second variant will ensure no false-sharing. The first variant may result in false sharing as the cache line size may really be hardware_destructive_interference_size in which case you will have false-sharing (under the assumption that hardware_constructive_interference_size < hardware_destructive_interference_size).

But in practice hardware_destructive_interference_size and hardware_constructive_interference_size will have the same value for most architectures. This is somewhat over engineered, given that neither constant provides you with the real L1 cache line size, but just with a compile-time guess.

Apparently, to get the best for small and large structs, should be aligned by `alignas(std::min(sizeof(A), std::hardware_constructive_interference_size))`. As `sizeof` cannot be used for incompletes, use of `sizeof` seem to involve extra wrapping, like `struct alignas(...sizeof(A)...) A_aligned : A {};` — Alex Guteniev, May 04 '20 at 17:33

Correct way to ensure sharing with std::hardware_constructive_interference_size

1 Answers1

Linked