what does the cmpq instruction do?

Question

I was reading the following definition for syscall:

.text
.globl syscall
.type   syscall,%function
.align 16
syscall:
    movq %rdi, %rax     /* Syscall number -> rax.  */
    movq %rsi, %rdi     /* shift arg1 - arg5.  */
    movq %rdx, %rsi
    movq %rcx, %rdx
    movq %r8, %r10
    movq %r9, %r8
    movq 8(%rsp),%r9    /* arg6 is on the stack.  */
    syscall         /* Do the system call.  */
    cmpq $-4095, %rax   /* Check %rax for error.  */
    jae __syscall_error     /* Branch forward if it failed.  */
    ret         /* Return to caller.  */

.size syscall,.-syscall

I saw it explained that the line cmpq $-4095 %rax determines whether %rax contains a value between -1 and -4095. How does it do that? What exactly does the cmpq instruction do?

Answer 1

cmpq $-4095, %rax compares the 64-bit register %rax with the immediate value -4095 - the value is sign-extended to 64-bits for the purposes of the comparison. i.e., -4095 has the 64-bit 2's complement representation: ffff ffff ffff f001

cmp instructions set the flags register as it would for a sub (subtract) of the second operand from the first - 'second' and 'first' being reversed in AT&T syntax. In effect the flags are set according to the result of: (RAX - (- 4095)) or (RAX + 4095), being the same in 2's complement.

One of the flags set is the carry flag (CF), which is set on (unsigned) overflow. The jae instruction (jump-if-above-or-equal) is actually an 'alias' for jnc (jump-if-not-carry). In other words, the branch is taken if (RAX + 4095) does not carry. In 2's complement, this will be true for values of RAX in the range: [-4095, -1]. (Keeping in mind how 2's complement arithmetic wraps).

---

The instructions, including cmp and jae (or j<cond>) are described in: Intel® 64 and IA-32 Architectures Software Developer’s Manual, Volume 2.

The [E]FLAGS register (and what the arithmetic flags denote) are described in section 3.4.3 of Intel® 64 and IA-32 Architectures Software Developer’s Manual, Volume 1.