I vaguely remember from conversations with colleagues that changing compiler flags can break ABI compatibility. I'm not sure if this is always the case, or if it applies to all flags.
Specifically, I'm wondering if optimization flags (e.g. gcc -O2, gcc-O3) or to be more precise, flags to do with SIMD vectorization (e.g. gcc -mavx2, or gcc -msse3) can cause ABI compatibility issues.
I'm writing a vision library that uses a 3rd party closed-source shared library (really, dlls, because we're on windows).
I'm thinking about doing proper aligned memory allocations and enabling vectorization through compiler flags (/ARCH:AVX2 e.g. for MSVC).
I'm wondering if this will break compatibility with the dlls supplied by the 3rd party vendor.
If it may, then there is a lot less to think about, at the design level. If it doesn't then I'll have to think about implementing my own memory allocators etc.
My expectation is that optimization flags only affect code generation and not memory layout, and my understanding is that ABI compatibility has to do with memory layout.