What's the point of adding ud2 after writing to [ds:0x0]?

Question

I was experimenting with GCC, trying to convince it to assume that certain portions of code are unreachable so as to take opportunity to optimize. One of my experiments gave me somewhat strange code. Here's the source:

#include <iostream>

#define UNREACHABLE {char* null=0; *null=0; return {};}

double test(double x)
{
    if(x==-1) return -1;
    else if(x==1) return 1;
    UNREACHABLE;
}

int main()
{
    std::cout << "Enter a number. Only +/- 1 is supported, otherwise I dunno what'll happen: ";
    double x;
    std::cin >> x;
    std::cout << "Here's what I got: " << test(x) << "\n";
}

Here's how I compiled it:

g++ -std=c++11 test.cpp -O3 -march=native -S -masm=intel -Wall -Wextra

And the code of test function looks like this:

_Z4testd:
.LFB1397:
        .cfi_startproc
        fld     QWORD PTR [esp+4]
        fld1
        fchs
        fld     st(0)
        fxch    st(2)
        fucomi  st, st(2)
        fstp    st(2)
        jp      .L10
        je      .L11
        fstp    st(0)
        jmp     .L7
.L10:
        fstp    st(0)
        .p2align 4,,10
        .p2align 3
.L7:
        fld1
        fld     st(0)
        fxch    st(2)
        fucomip st, st(2)
        fstp    st(1)
        jp      .L12
        je      .L6
        fstp    st(0)
        jmp     .L8
.L12:
        fstp    st(0)
        .p2align 4,,10
        .p2align 3
.L8:
        mov     BYTE PTR ds:0, 0
        ud2         // This is redundant, isn't it?..
        .p2align 4,,10
        .p2align 3
.L11:
        fstp    st(1)
.L6:
        rep; ret

What makes me wonder here is the code at .L8. Namely, it already writes to zero address, which guarantees segmentation fault unless ds has some non-default selector. So why the additional ud2? Isn't writing to zero address already guaranteed crash? Or does GCC not believe that ds has default selector and tries to make a sure-fire crash?

Answer 1

So, your code is writing to address zero (NULL) which in itself is defined to be "undefined behaviour". Since undefined behaviour covers anything, and most importantly for this case, "that it does what you may imagine that it would do" (in other words, writes to address zero rather than crashing). The compiler then decides to TELL you that by adding an UD2 instruction. It's also possible that it is to protect against continuing from a signal handler with further undefined behaviour.

Yes, most machines, under most circumstances, will crash for NULL accesses. But it's not 100% guaranteed, and as I said above, one can catch segfault in a signal handler, and then try to continue - it's really not a good idea to actually continue after trying to write to NULL, so the compiler adds UD2 to ensure you don't go on... It uses 2 bytes more of memory, beyond that I don't see what harm it does [after all, it's undefined what happens - if the compiler wished to do so, it could email random pictures from your filesystem to the Queen of England... I think UD2 is a better choice...]

It is interesting to spot that LLVM does this by itself - I have no special detection of NIL pointer access, but my pascal compiler compiles this:

program p;

var
   ptr : ^integer;

begin
   ptr := NIL;
   ptr^ := 42;
end.

into:

0000000000400880 <__PascalMain>:
  400880:   55                      push   %rbp
  400881:   48 89 e5                mov    %rsp,%rbp
  400884:   48 c7 05 19 18 20 00    movq   $0x0,0x201819(%rip)        # 6020a8 <ptr>
  40088b:   00 00 00 00 
  40088f:   0f 0b                   ud2    

I'm still trying to figure out where in LLVM this happens and try to understand the purpose of the UD2 instruction itself.

I think the answer is here, in llvm/lib/Transforms/Utils/Local.cpp

void llvm::changeToUnreachable(Instruction *I, bool UseLLVMTrap) {
  BasicBlock *BB = I->getParent();
  // Loop over all of the successors, removing BB's entry from any PHI
  // nodes.
  for (succ_iterator SI = succ_begin(BB), SE = succ_end(BB); SI != SE; ++SI)
    (*SI)->removePredecessor(BB);

  // Insert a call to llvm.trap right before this.  This turns the undefined
  // behavior into a hard fail instead of falling through into random code.
  if (UseLLVMTrap) {
    Function *TrapFn =
      Intrinsic::getDeclaration(BB->getParent()->getParent(), Intrinsic::trap);
    CallInst *CallTrap = CallInst::Create(TrapFn, "", I);
    CallTrap->setDebugLoc(I->getDebugLoc());
  }
  new UnreachableInst(I->getContext(), I);

  // All instructions after this are dead.
  BasicBlock::iterator BBI = I->getIterator(), BBE = BB->end();
  while (BBI != BBE) {
    if (!BBI->use_empty())
      BBI->replaceAllUsesWith(UndefValue::get(BBI->getType()));
    BB->getInstList().erase(BBI++);
  }
}

In particular the comment in the middle, where it says "instead of falling through to the into random code". In your code there is no code following the NULL access, but imagine this:

void func()
{
    if (answer == 42)
    {
     #if DEBUG
         // Intentionally crash to avoid formatting hard disk for now

         char *ptr = NULL;
         ptr = 0;
    #endif
         // Format hard disk. 
         ... some code to format hard disk ... 
    }
    printf("We haven't found the answer yet\n");
    ... 
}

So, this SHOULD crash, but if it doesn't the compiler will ensure that you do not continue after it... It makes UB crashes a little more obvious (and in this case prevents the hard disk from being formatted...)

I was trying to find out when this was introduced, but the function itself originates in 2007, but it's not used for exactly this purpose at the time, which makes it really hard to figure out why it is used this way.