0

I read on some blogs online you can use int 2Eh or sysenter instead of syscall in an asm file. So I tried both of these but it doesn't work for me to run the program. For int 2Eh, it compiles but program doesn't run and do what it should do(which it does if I use "syscall" instead). And for sysenter, it doesn't even compile because compiler can't recognize the word syscall.

I am using latest Visual Studio and my project is an 64-bit exe file. Anyone have ideas what I can try?

Peter Cordes
  • 328,167
  • 45
  • 605
  • 847
  • Pretty sure you shouldn't use `int 2eh` in 64-bit Windows code. At best, it would be the 32-bit ABI. As for `sysenter`, Intel's manual says it's valid in 32-bit mode (https://www.felixcloutier.com/x86/sysenter), although again you might not be able to actually execute it without faulting. But if your assembler doesn't even assemble it, use a better assembler (e.g. NASM, which has no problem with it in 64-bit mode.) – Peter Cordes Dec 20 '21 at 11:10
  • @PeterCordes A friend told me ue used int 2EH in 64-bit mode. Is there any alternative to syscall that works in 64-bit when using Visual Studio as a compiler? –  Dec 20 '21 at 13:43
  • I'd assume that `int 2Eh` isn't a drop-in replacement, it might use a different ABI like is the case on Linux for `int 0x80` in 64-bit code where it uses different registers. ([What happens if you use the 32-bit int 0x80 Linux ABI in 64-bit code?](https://stackoverflow.com/q/46087730)). So check the reverse-engineered calling-convention details (maybe https://j00ru.vexillium.org/syscalls/nt/32/ is relevant? IDK, I don't know Windows that well) or wait for for someone else to answer. – Peter Cordes Dec 20 '21 at 14:10
  • Also, obviously it's not Visual Studio that's important, it's that you're making an x64 Windows executable. (And if you really wanted to try sysenter in MASM, you could encode it manually with `db 0Fh, 34h`.) – Peter Cordes Dec 20 '21 at 14:11
  • @PeterCordes Okay I see. I tried now to replace the syscall with "db 0Fh, 34h" in my asm file and recompiled the 64-bit code, but it doesn't run the code so that didn't work either. –  Dec 20 '21 at 14:25
  • I didn't expect it would work as a drop-in replacement; sysenter is pretty different from syscall, e.g. not saving the user-space stack pointer, so user-space needs to copy it somewhere so the kernel or user-space can restore it later. (e.g. look at Linux 32-bit VDSO code for the user-space side of its sysenter dance.) But that's how you can get a `sysenter` into your machine code, so it worked for doing that. – Peter Cordes Dec 20 '21 at 14:29
  • The system calls in Windows change - some of them even during Windows updates! This means that a program directly using `int 2Eh` or `syscall` may work completely differently before and after a Windows update. Microsoft might even decide to remove both `syscall` and `int 2Eh` support in the next Windows update and use `int 3Fh` instead! Better don't use `syscall`and `int 2Eh` in Windows programs for this reason. – Martin Rosenau Dec 21 '21 at 10:30

0 Answers0