Why does enabling undefined behaviour sanitization interfere with optimizations?

Question

Consider the following code:

#include <string_view>

constexpr std::string_view f() { return "hello"; }

static constexpr std::string_view g() {
    auto x = f();
    return x.substr(1, 3);
}

int foo() { return g().length(); }

If I compile it with GCC 10.2, and flags --std=c++17 -O1, I get:

foo():
        mov     eax, 3
        ret

also, to my knowledge, this code does not suffer from any undefined behavior issues.

However - if I add the flag -fsanitize=undefined, the compilation result is:

.LC0:
        .string "hello"
foo():
        sub     rsp, 104
        mov     QWORD PTR [rsp+80], 5
        mov     QWORD PTR [rsp+16], 5
        mov     QWORD PTR [rsp+24], OFFSET FLAT:.LC0
        mov     QWORD PTR [rsp+8], 3
        mov     QWORD PTR [rsp+72], 4
        mov     eax, OFFSET FLAT:.LC0
        cmp     rax, -1
        jnb     .L4
.L2:
        mov     eax, 3
        add     rsp, 104
        ret
.L4:
        mov     edx, OFFSET FLAT:.LC0+1
        mov     rsi, rax
        mov     edi, OFFSET FLAT:.Lubsan_data154
        call    __ubsan_handle_pointer_overflow
        jmp     .L2
.LC1:
        .string "/opt/compiler-explorer/gcc-10.2.0/include/c++/10.2.0/string_view"
.Lubsan_data154:
        .quad   .LC1
        .long   287
        .long   49

See this on Compiler Explorer.

My question: Why should the sanitization interfere with the optimization? Especially since the code doesn't seem to have any UB hazards...

Notes:

I suspect a GCC bug, but maybe I have the wrong perception of what the UBsan does.
Same behavior if I set -O3.
With no optimization flags, the longer code is produced both with and without sanitization.
If you declare x to be a constexpr variable, the sanitization doesn't prevent the optimization.
Same behavior with C++17 and C++20.
With Clang, you get this discrepancy as well, but only with a higher optimization setting (e.g. -O3).

Asan is not the same as UBsan. Anyway, I guess it's just the addition of instrumenting that does this, and the compile either doesn't or can't optimise around that. — underscore_d, Oct 23 '20 at 16:06
My guess is that enabling sanitization disabled the data flow analysis. So instead of determining that there can't be any buffer overflow at compile time, it generates code to check for it at runtime. — Barmar, Oct 23 '20 at 16:12
@Barmar: But why should enabling sanitization disable the data flow analysis? ... is my question, really. — einpoklum, Oct 23 '20 at 16:17
@einpoklum Your original title said "address sanitization", but the question is about undefined behavior sanitization. — Barmar, Oct 23 '20 at 16:18
It didn't interfere with optimizations: The `mov eax, 3` is still there, and there are no calls to `f` or `g`. It just added instrumentation (which is admittedly unneeded here).. — interjay, Oct 23 '20 at 16:24

Dan M. · Accepted Answer · 2020-10-23T17:07:35.083

Sanitizers add necessary instrumentation to detect violations at run-time. That instrumentation may prevent the function from being computed at compile-time as an optimization by introducing some opaque calls/side-effects that wouldn't be present there otherwise.

The inconsistent behavior you see is because g().length(); call is not done in constexpr context, so it's not required (well, "not expected" would be more accurate) to be computed at compile-time. GCC likely has some heuristics to compute constexpr functions with constexpr arguments in regular contexts that don't trigger once sanitizers get involved by either breaking the constexpr-ness of the function (due to added instrumentation) or one of the heuristics involved.

Adding constexpr to x makes f() call a constant expression (even if g() is not), so it's compiled at compile-time so it doesn't need to be instrumented, which is enough for other optimizations to trigger.

One can view that as a QoI issue, but in general it makes sense as

constexpr function evaluation can take arbitrarily long, so it's not always preferable to evaluate everything at compile time unless asked to
you can always "force" such evaluation (although the standard is somewhat permissive in this case), by using such functions in constant expressions. That'd also take care of any UB for you.

score 5 · Answer 2 · answered Oct 23 '20 at 20:07

Especially since the code doesn't seem to have any UB hazards

f() returns a std::string_view which contains a length and a pointer. The call to x.substr(1, 3) requires adding one to that pointer. That technically may overflow. That is the potential UB. Change 1 to 0 and see the UB code go away.

We know that [ptr, ptr+5] are valid, so the conclusion is that gcc fails to propagate that knowledge of the value range, despite aggressive inlining and other simplification.

I can't find a directly related gcc bug, but this comment seems interesting:

[VRP] does an incredibly bad job at tracking pointer ranges where it simply prefers to track non-NULL.

score 3 · Answer 3 · answered Oct 23 '20 at 16:15

Undefined behavior sanitizers are not a compiler-time-only mechanism (emphasis not in the original; and the quote is about clang but it applies to GCC as well):

UndefinedBehaviorSanitizer (UBSan) is a fast undefined behavior detector. UBSan modifies the program at compile-time to catch various kinds of undefined behavior during program execution.

So, instead of the original program - what actually gets compiled is a program with some additional "instrumentation" that you can see in the longer compiled code, e.g.:

Additional instructions which the original program would should not be able to get to.
An indication of where in the standard-library code the inappropriately-executed code is related.

Apparently, GCC's optimizers can't detect that there actually won't be any undefined behavior, and drop the unused code.

Why does enabling undefined behaviour sanitization interfere with optimizations?

3 Answers3