Do C compilers optimize away functions in assembly so they minimize use of the stack?

Question

I am starting to learn assembly (x86-64 in NASM on OSX), and am now exploring how functions look in it.

Most resources explaining how "calling conventions" work show examples along the lines of this:

// c code
MyFunction1(a, b);

// assembly code
main:
  push a
  push b
  push rbp ; save frame pointer on stack
  mov rsp, rbp  ; save stack pointer in frame pointer
  xor rax, rax  ; set function return value to 0.
  call _MyFunction
  mov rbp, rsp  ; restore stack pointer
  pop rbp ; restore frame pointer
  ret ; return to calling function

(I just made that up after combining several resources, so there are probably many problems with that, but that's outside the main question.).

The gist of what calling conventions like the cdecl calling convention, is that:

You push the arguments in reverse order from how they appear in the C code.
You then save a reference to both the frame and stack pointers (I'm guessing so you can do this recursively, but haven't got that far yet to know).
Then you compute whatever you need to in your function.
And after that, pop the stack and frame pointers off the stack

So, in hopes of getting some more practical experience of working with the stack and function calling conventions in assembly, I was hoping to see how existing C compilers converted function calls into assembly (gcc and clang, using this which is great). But, I am not seeing that calling convention pattern (which every resource I've seen so far has said is the way to do it)!

Check it out, here is a relatively complex bit of assembly code generated from some C:

https://gist.github.com/lancejpollard/a1d6a9b4820473ed8797

Looking through that C code, there are a couple levels of nested function calls. But the output assembly code doesn't show that push/pop stack pattern!

So the question is, are these compilers just optimizing the assembly so as to avoid doing that? (Because, that example C code, while having some nested functions, is still pretty simple so I imagine the compiler can precompute a lot of stuff). Or am I missing something?

Compilers optimise for all sorts of reasons. Stack usage is one but they also look at execution speed, overall code size, register usage and a host of other things. A calling convention is useful when you're linking code together that's compiled separately, but if the compiler can optimise the use of functions in a single compilation a calling convention is redundant. — , Dec 22 '14 at 03:32
Your C code doesn't really match your assembly code. If you think they are equivalent, that may be part of your misunderstanding. — Dwayne Towell, Dec 22 '14 at 03:43
@DwayneTowell interesting, that is what you get if you paste that C code into http://gcc.godbolt.org/. I forget which compiler gave those exact results (there are over 10 compilers to choose from on that site), but they were all fairly similar. — Lance, Dec 22 '14 at 03:48
Remember main() is a function too. Some of the code is to manage the stack as a *called* function, and some of the code is to *call* another function. You seem to mixing up those two ideas. — Dwayne Towell, Dec 22 '14 at 03:54

Brendan · Accepted Answer · 2014-12-22T04:57:12.687

In general:

if a function may be called by something in a different object file (e.g. the function isn't static and you are creating an object file) then the compiler has to respect the calling conventions (so that linking can work).
if you're compiling for debugging then a typical debugger will use the stack frames to find things (input parameters, local variables, etc) and performance isn't a high priority, so respecting the calling conventions is a good idea.

For all other cases the compiler is free to completely ignore calling conventions and do whatever it thinks is more efficient (including passing parameters in registers and not using the stack or frame pointer, and also including inlining functions completely).

However:

for some cases (e.g. function pointers and functions with a variable number of arguments) the compiler may not be able to do much and might just use the standard calling conventions (even when it doesn't strictly need to).
it is possible for the compiler to generate 2 or more different versions of a function (e.g. one that respects the calling convention that code in other object files can use, plus another optimised version that doesn't respect calling convention).
the linker may do optimisations that the compiler couldn't (link time optimisation), including modifying the calling conventions for functions that the compiler couldn't optimise (as the compiler didn't have enough knowledge of potential callers) but the linker can (as the linker "sees" the whole program).

score 0 · Answer 2 · answered Dec 22 '14 at 04:50

you can get the C compiler to output assember source code instead of linkable objects. On linux, cc -S file.c will create a file.s containing the assembly code. Try a small test function, and various levels of optimization. It's quite easy to recognize the calling convention, it's very similar to your pseudocode.

file.c:

foo(int x)
{
    return x+1;
}

file.s:

        .file   "t.c"
        .text
.globl foo
        .type   foo, @function
foo:
        pushl   %ebp
        movl    %esp, %ebp
        movl    8(%ebp), %eax
        addl    $1, %eax
        popl    %ebp
        ret
        .size   foo, .-foo
        .ident  "GCC: (Ubuntu 4.4.3-4ubuntu5.1) 4.4.3"
        .section        .note.GNU-stack,"",@progbits

Do C compilers optimize away functions in assembly so they minimize use of the stack?

2 Answers2

Linked