Segmentation fault when using DB (define byte) inside a function

Question

I'm trying to define a byte in Assembly language inside my .text section. I know data should go to the .data section but I was wondering why it gives me a segmentation fault when I do it. If I define the byte inside .data, it doesn't give me any errors, unlike .text. I am using a Linux machine running Mint 19.1 and using NASM + LD to compile and link the executable.

This runs without segmentation faults:

global _start
section .data
db 0x41
section .text
_start:
    mov rax, 60    ; Exit(0) syscall
    xor rdi, rdi
    syscall

This gives me a segfault:

global _start
section .text
_start:
    db 0x41
    mov rax, 60     ; Exit(0) syscall
    xor rdi, rdi
    syscall

I'm using the following script to compile and link it:

nasm -felf64 main.s -o main.o
ld main.o -o main

I expect the program to work without any segmentation faults, but it doesn't when I use DB inside .text. I suspect that .text is readonly and that may be the reason of this problem, am I correct? Can someone explain to me why my second code example segfaults?

Answer 1

If you tell the assembler to assemble arbitrary bytes somewhere, it will. db is a pseudo-instruction that emits bytes, so mov eax, 60 and db 0xb8, 0x3c, 0, 0, 0 are exactly equivalent as far as NASM is concerned. Either one will emit those 5 bytes into the output at the current position.

If you don't want your data decoded as (part of) instructions, don't put it where it will be reached by execution.

---

Since you're using NASM¹, it optimizes mov rax,60 into mov eax,60, so the instruction doesn't have the REX prefix you'd expect from the source.

Your manually-encoded REX prefix for mov changes it into a mov to R8D instead of EAX:
41 b8 3c 00 00 00 mov r8d,0x3c

(I checked with objdump -drwC -Mintel instead of looking up which bit is which in the REX prefix. I only remember that REX.W is 0x48. But 0x41 is a REX.B prefix in x86-64).

So instead of making a sys_exit system call, your code runs syscall with EAX=0, which is __NR_read. (The Linux kernel zeros all the registers other than RSP before process startup, and in a statically-linked executable, _start is the true entry point with no dynamic linker code running first. So RAX is still zero).

$ strace ./rex 
execve("./rex", ["./rex"], 0x7fffbbadad60 /* 54 vars */) = 0
read(0, NULL, 0)                        = 0
--- SIGSEGV {si_signo=SIGSEGV, si_code=SEGV_MAPERR, si_addr=NULL} ---
+++ killed by SIGSEGV (core dumped) +++

And then execution falls through into whatever is after syscall, which in this case is 00 00 bytes that decode as add [rax], al, and thus segfault. You would have seen this if you'd run your code inside GDB.

---

Footnote 1: If you'd used YASM which doesn't optimize to 32-bit operand size:

Intel's manuals say that it's illegal to have 2 REX prefixes on one instruction. I expected an illegal-instruction fault (#UD machine exception -> kernel delivers SIGILL), but my Skylake CPU ignores the first REX prefix and decodes it as mov rax, sign_extended_imm32.

Single-stepping, it's treated as one long instructions, so I guess Skylake chooses to handle it like other cases of multiple prefixes, where only the last one of a type has an effect. (But remember this is not future-proof, other x86 CPUs could handle it differently.)

---

Related / same bug in other situations:

• Assembly (x86): <label> db 'string',0 does not get executed unless there's a jump instruction in a BIOS MBR boot sector
• Unknown opcode skipped: 66, not 8086 instruction - not supported yet in EMU8086