13

I know that MSVC compiler in x64 mode does not support inline assembly snippets of code, and in order to use assembly code you have to define your function in some external my_asm_funcs.asm file like that:

my_asm_func PROC
    mov rax, rcx
    ret
my_asm_func ENDP

And then in your .c or .h file you define a header for the function like that:

int my_asm_func(int x);

Although that solution answers many concerns, but I am still interested in making that assembly code function to be inline, in other words - after compilation I don't want any "calls" to my_asm_func, I just want this piece of assembly to be glued into my final compiled code. I tried declaring the function with inline and __forceinline keywords, but nothing seems to be helping. Is there still any way to do what I want?

Peter Cordes
  • 328,167
  • 45
  • 605
  • 847
user1483597
  • 540
  • 2
  • 4
  • 20
  • 2
    Have you searched through compiler intrinsics? It might be the chance, that there is something, that covers `my_asm_func`'s functionality. In general, mixing your own assembly inline with compiler that makes a lot of optimizations (depending upon level of optimization) may be tricky to get working, not to mention its maintenance. – Grzegorz Szpetkowski Dec 18 '16 at 12:10
  • 1
    No, not possible. See this [answer to a similar question](http://stackoverflow.com/a/5542379/597607) for some reasons. – Bo Persson Dec 18 '16 at 12:10

3 Answers3

15

No, there is no way to do what you want.

Microsoft's compiler doesn't support inline assembly for x86-64 targets, as you said. This forces you to define your assembly functions in an external code module (*.asm), assemble them with MASM, and link the result together with your separately-compiled C/C++ code.

The required separation of steps means that the C/C++ compiler cannot inline your assembly functions because they are not visible to it at the time of compilation.

Even with link-time code generation (LTCG) enabled, your assembly module(s) will not get inlined because the linker simply doesn't support this.

There is absolutely no way to get assembly functions written in a separate module inlined directly into C or C++ code.

There is no way that the inline or __forceinline keywords could do anything. In fact, there's no way that you could use them without a compiler error (or at least a warning). These annotations have to go on the function's definition (which, for an inline function, is the same as its declaration), but you can't put it on the function's definition, since that's defined in a separate *.asm file. These aren't MASM keywords, so trying to add them to the definition would necessarily result in an error. And putting them on the forward declaration of the assembly function in the C header is going to be similarly unsuccessful, since there's no code there to inline—just a prototype.

This is why Microsoft recommends using intrinsics. You can use these directly in your C or C++ code, and the compiler will emit the corresponding assembly code automatically. Not only does this accomplish the desired inlining, but intrinsics even allow the optimizer to function, further improving the results. No, intrinsics do not lead to perfect code, and there aren't intrinsics for everything, but it's the best you can do with Microsoft's compiler.

Your only other alternative is to sit down and play with various permutations of C/C++ code until you get the compiler to generate the desired object code. This can be very powerful in cases where intrinsics are not available for the instructions that you wish to be generated, but it does take a lot of time spent fidgeting, and you'll have to revisit it to make sure it continues to do what you want when you upgrade compiler versions.

Cody Gray - on strike
  • 239,200
  • 50
  • 490
  • 574
  • 3
    You may be able to mark the function `naked`. That may sidestep the function call and simply perform a jump to the free standing asm code. Also see [How to do a naked function and inline assembler in x64 Visual C++](https://stackoverflow.com/q/26637755/608639). – jww Apr 25 '19 at 02:20
  • 2
    @jww `__declspec(naked)` does not cause the function to be inlined, either. In fact, if I'm remembering my testing correctly, the `naked` annotation actually makes the optimizer *even more* pessimistic. Also...this question is about x86-64 targets, and `__declspec(naked)` doesn't even exist for x86-64 targets. – Cody Gray - on strike Apr 25 '19 at 02:49
4

Since the title mentions Visual Studio and not MSVC, I recommend installing Clang via the Visual Studio Installer. It can be used just like MSVC without needing to configure custom build tasks or anything and it supports inline assembly with Intel syntax and variables as operands.

Just select "LLVM (clang-cl)" in Platform Toolset from the General section of the property pages in your project and you're good to go.

Big Temp
  • 434
  • 4
  • 12
0

Yes you can. Assemble your procedure as shellcode and extract the bytes, then include it in a buffer with RWX memory protection in your code. Call the code.

Toxcique
  • 49
  • 1
  • 4
  • 1
    The call site will still use a `call` instruction. That avoids a separate .asm file, but it has no benefit for performance - it doesn't inline the way `uint64_t foo(uint64_t x) { return x;}` can. (And in fact costs performance since you need to call through a function pointer instead of a normal `call rel32`, as well as allocate the buffer, unlike a separate `.asm` file that just gets assembled and linked normally into your executable, next to the compiler-generated code). – Peter Cordes May 25 '21 at 20:46
  • 1
    It's also worse for maintainability; editing the asm requires re-generating the `static char machine_code[] = { ... };` array or whatever you use, instead of just editing the asm source in a separate file. Still, worth mentioning as a hack to get asm into a single file, e.g. for online IDEs or other limited cases, but only if your answer makes it clear what it does / doesn't do. (i.e. answering a somewhat different question than what was asked.) – Peter Cordes May 25 '21 at 20:49