Doesn't 64-bit kernel support IA-32 application with .code64 assembler?

Question

x86 application(build with gcc -m32) supports Assembly 64-bit code with .code64, this means that an x86 application could use a 64-bit register. But from kernel side, the application just a IA-32 application.

For example, I can link below symbol 64bit_test to an x86 application.

ENTRY(64bit_test)
    .code64;
    push %r8
    push %r12
END(64bit_test)

When the kernel setup signal handler, the kernel only saved 32-bit registers without 64-bit registers, does 64-bit register context missing? I think this is incorrect because 64-bit register is used and should be saved and restored later.

if (is_ia32_frame(ksig)) {
        if (ksig->ka.sa.sa_flags & SA_SIGINFO)
            return ia32_setup_rt_frame(usig, ksig, cset, regs);
        else
            return ia32_setup_frame(usig, ksig, cset, regs);
    } else if (is_x32_frame(ksig)) {
        return x32_setup_rt_frame(ksig, cset, regs);
    } else {
        return __setup_rt_frame(ksig->sig, ksig, set, regs);
    }


static int ia32_setup_sigcontext(struct sigcontext_32 __user *sc,
                 void __user *fpstate,
                 struct pt_regs *regs, unsigned int mask)
{
    int err = 0;

    put_user_try {
        put_user_ex(get_user_seg(gs), (unsigned int __user *)&sc->gs);
        put_user_ex(get_user_seg(fs), (unsigned int __user *)&sc->fs);
        put_user_ex(get_user_seg(ds), (unsigned int __user *)&sc->ds);
        put_user_ex(get_user_seg(es), (unsigned int __user *)&sc->es);

        put_user_ex(regs->di, &sc->di);
        put_user_ex(regs->si, &sc->si);
        put_user_ex(regs->bp, &sc->bp);
        put_user_ex(regs->sp, &sc->sp);
        put_user_ex(regs->bx, &sc->bx);
        put_user_ex(regs->dx, &sc->dx);
        put_user_ex(regs->cx, &sc->cx);
        put_user_ex(regs->ax, &sc->ax);
        put_user_ex(current->thread.trap_nr, &sc->trapno);
        put_user_ex(current->thread.error_code, &sc->err);
        put_user_ex(regs->ip, &sc->ip);
        put_user_ex(regs->cs, (unsigned int __user *)&sc->cs);
        put_user_ex(regs->flags, &sc->flags);
        put_user_ex(regs->sp, &sc->sp_at_signal);
        put_user_ex(regs->ss, (unsigned int __user *)&sc->ss);

        put_user_ex(ptr_to_compat(fpstate), &sc->fpstate);

        /* non-iBCS2 extensions.. */
        put_user_ex(mask, &sc->oldmask);
        put_user_ex(current->thread.cr2, &sc->cr2);
    } put_user_catch(err);

    return err;
}

I expect 64-bit registers r8 to r15 should be saved in sigcontext and later restored, but from the code, r8 to r15 is missing.

score 3 · Accepted Answer · edited Jun 20 '20 at 09:12

TL:DR: no, that's not what .code64 does, and no Linux doesn't support 32-bit processes that far-jump to 64-bit user-space.

.code64 just lets you put 64-bit machine code inside a 32-bit object file / executable. e.g. if you wanted to write a 32-bit program that patched a 64-bit executable, and wanted to have the assembler generate that data for you even though it would never be executed inside the 32-bit program.

Or if you were writing your own kernel that started in 16 or 32-bit mode, and switched to 64-bit mode, you'd use .code64 for the part that your kernel jumps with CS referring to a 64-bit code segment.

Decoding machine code as 64-bit instead of 32-bit requires putting the CPU in a different mode. x86 machine code does not support mixing 32 and 64-bit machine code without a mode-switch. There isn't enough coding space left for that. The encodings are very similar but with some opcodes having a different default operand-size in 64-bit mode (e.g. stack ops) e.g. push %eax and push %rax have the same 1-byte opcode.

Your .code64; ; push %r8 test actually creates 32-bit machine code for inc %eax (the REX prefix) and push %eax. Yes it assembles and runs, but as different instructions. Single-step it with GDB in layout reg to see disassembly according to the actual mode the CPU is in, not source.

The differences include 64-bit long mode repurposing 1-byte inc/dec (0x40..4f) opcodes as REX prefixes. e.g. x86-32 / x86-64 polyglot machine-code fragment that detects 64bit mode at run-time?

Note that this is very different from 16 vs. 32. 16-bit code can use an operand-size prefix in 16-bit mode to access 32-bit registers and addressing modes. e.g. mov eax, 1234 assembles just fine in .code16 (with an operand-size prefix), or in .code32 (without a prefix).

But you can't do add rax, rdx outside of .code64 because there's no way to run it without switching the CPU to a different mode. (Modes are selected by the GDT / LDT entry that CS is pointing to).

You can in theory jmpl (far jmp) in user-space to a different code segment in your user-space process to switch from "compat mode" (32-bit mode under a 64-bit kernel) to full 64-bit mode. You'd have to know what CS value to use, but most OSes have some "well-known" constant values for their 32-bit and 64-bit user-space (CPL=3) code segments.

If that sounds incredibly arcane and complex, yeah that's my point.

There's basically zero support (from OS system calls and context switches, dynamic linker, and toolchains) for switching modes inside a process. It's generally a terrible idea, don't do it.

e.g. as you noticed, the kernel only saves / restores the legacy IA32 state for a process that started as 32-bit when delivering a signal, so if it had far-jumped to 64-bit user-space the signal handler would corrupt high registers. (r8..r11 are call-clobbered in the x86-64 System V ABI).

Semi-related: What happens if you use the 32-bit int 0x80 Linux ABI in 64-bit code?

Very nice answer! I think the OSes didn't want to support far jumping/calling because there is no high level support for it (to account for the ABIs and low 4GiB pointers) and because always saving the whole 64-bit context is slow. There's nothing too difficult that prevents an OS to allow mixing 32 and 64 bit code but an ABI like X32 is probably the best of the two worlds if one doesn't want the addressing nuisances of the long mode :) — Margaret Bloom, Aug 23 '19 at 15:01
@MargaretBloom: exactly. An ILP32 ABI in 64-bit mode is usually better than switching to 32-bit mode if you want to use 32-bit pointers. The only reason you might want this is legacy baggage like 32-bit binary-only libraries. (Which is something that's best avoided in the first place.) — Peter Cordes, Aug 23 '19 at 16:11
Thsnks Peter and you all, you solved my doubts, this is a satisfactory answer. — taoozh, Aug 26 '19 at 05:19
@Peter Cordes: no matter application is i386, x32, or x64, when they run on 64bit, kernel always save the whole 64bit context when into kernel and restore whole 64bit context when back to user space, this is right? — taoozh, Aug 26 '19 at 07:15
@taoozh: The `int 0x80` kernel entry point pushes zeros as the saved values for r8..r11. So when used from a 64-bit process, it zeros some registers. but that's synchronous, and your own fault [if you use int 0x80 in 64-bit code](https://stackoverflow.com/questions/46087730/what-happens-if-you-use-the-32-bit-int-0x80-linux-abi-in-64-bit-code). But I assume interrupt entry points always save the full x86-64 context instead of detecting what task was running previously. FP/SIMD context is saved with `xsaveopt` or similar, so x/ymm8-15 presumably get saved even for 32-bit processes. — Peter Cordes, Aug 26 '19 at 07:38

Doesn't 64-bit kernel support IA-32 application with .code64 assembler?

1 Answers1