2

In many functions the function prologue starts with sub $0x1c,%esp, which is later removed by add $0x1c,%esp. The function can be as simple as doing virtually nothing. This space is never used however, and has become an annoyance to work with as I need full control over the stack of my functions. Any idea what causes this, and how can I remove it? I'm compiling with -O2 -Wall -c -m32 -fno-stack-protector -fno-defer-pop -fno-builtin -march=i386.

Seralize
  • 1,117
  • 11
  • 27
  • 4
    "I need full control over the stack of my functions" - curious: why? That's precisely what the compiler is there for. – Oliver Charlesworth Mar 05 '16 at 20:41
  • 1
    Some ABIs mandate some red zone on the call stack. What is your ABI and operating system? – Basile Starynkevitch Mar 05 '16 at 20:42
  • 2
    There's also the issue of alignment. – user3386109 Mar 05 '16 at 20:43
  • @OliverCharlesworth I'm writing a syscall mechanism and I'm controlling which return addresses remain on the stack before dispatching. It then gets troublesome when I try to return to an address, but the stack has been modified by GCC and I need to ret before reaching the "add $0x1c,%esp". – Seralize Mar 05 '16 at 20:44
  • 4
    @Seralize: You cannot make assumptions that the compiler does not alter the stack pointer or modify any registers. If you need to do this you have to write the whole function in asm, not as inline asm. – R.. GitHub STOP HELPING ICE Mar 05 '16 at 20:46
  • @R I'm writing it as a mix of assembly and C and had hoped I could avoid writing all of it in assembly, or most of it. – Seralize Mar 05 '16 at 20:47
  • @BasileStarynkevitch I'm on Ubuntu (XUbuntu) 14.04.3 LTS. I don't know what ABI I'm using, it's for an operating system image. – Seralize Mar 05 '16 at 20:52
  • That space is added to protect stack from local variables access out of bounds (typically is added when local arrays are created). As already said you cannot make general assumptions about stack frame creation, but the for specific compiler/system you can consider stack frame standardized. Typically the function prologue saves stack pointer to the base pointer then reserves space on the stack for local variables and safeguard. Normally you can refer to ebp registers on X86 CPU's as base pointer. – Frankie_C Mar 05 '16 at 20:59
  • @Frankie_C It makes no sense for a compiler to allocate extra space as a safeguard against out-of-bounds accesses, nor have I ever heard of a compiler doing such a thing. More likely the space is allocated for stack alignment purposes. – Tom Karzes Mar 05 '16 at 21:42
  • @TomKarzes - Isn't that exactly what GCC's stack-protector mode does? – Oliver Charlesworth Mar 05 '16 at 21:57
  • @TomKarzes See https://en.wikipedia.org/wiki/Buffer_overflow_protection on GNU Compiler Collection implementation: "From 2001 to 2005, IBM developed GCC patches for stack-smashing protection, known as ProPolice.[13] It improved on the idea of StackGuard by placing buffers after local pointers and function arguments in the stack frame. This helped avoid the corruption of pointers, preventing access to arbitrary memory locations". But this is used on almost all compilers (see https://msdn.microsoft.com/en-us/library/8dbf701c.aspx for MS). – Frankie_C Mar 05 '16 at 22:10
  • @Frankie_C Interesting. I tend not to like to de-optimize my code to avoid bugs that I don't introduce. In any case, OP lists `-fno-stack-protector` as one of the compiler options specified, so that can't apply in this instance. – Tom Karzes Mar 05 '16 at 23:56
  • @TomKarzes The stack probing, used also in the stack protector (`-fno-stack-protector`). should be on by default actually, and is used this way. You can make some control disassembling a sample. Now I remember the switch that can disable it should be `-fstack-check`. Anyway **studying the prologue the OP can use the base pointer value to unroll the stack frame** whichever is the space reserved on function enter. – Frankie_C Mar 06 '16 at 00:19
  • Instead of `-fno-builtin`, you should maybe be using `-ffreestanding`. This implies `-fno-builtin`. I'm not sure if there's a nice way to still get the optimized handling of `alloca`, `memcpy`, and stuff like that. – Peter Cordes Mar 06 '16 at 02:04
  • What gcc version? Can you post an example function? gcc 5.3 doesn't do anything weird like that for [`int foo (int x) { return x*2; }`](http://goo.gl/rRezxH). – Peter Cordes Mar 06 '16 at 02:06

1 Answers1

0

It would help to see more code but it's to do with stack alignment. You can mess around with this in GCC using -mpreferred-stack-boundary. The default on my machine is 4 so things are aligned to 16 bytes. I'm pretty confident this has nothing to do with stack protection. To see what happens with stack protection compile the program with -fno-stack-check and -fstack-check and diff the results. You'll see something like 

>   orl $0, -12300(%esp)
Harry
  • 11,298
  • 1
  • 29
  • 43
  • Keeping the stack aligned only explains adjustments of less than 0x10, and then only in non-leaf functions. – Peter Cordes Mar 06 '16 at 02:13
  • @PeterCordes I'm not sure I agree. Cache lines are one reason to force something to align at 32/64 or 128 bytes. The subtraction from the stack pointer includes any necessary stack space needed for locals etc including any padding the compiler deems necessary. Without knowing more about what platform the OP's using and some code it's hard to tell what's really going on but I'm sticking with alignment as the reason. Note, heres a bug filed on Clang discussing 32 byte stack alignment on gcc 4.8... https://llvm.org/bugs/show_bug.cgi?id=18006 32 byte byte alignment is not unheard. – Harry Mar 06 '16 at 06:23
  • Yeah, I wondered if the OP was looking at a function that did use *some* of the space it reserved, but I figured I'd take his claim literally and assume he was seeing `sub esp, 0x1c` in a function that didn't use the stack other than to load args and return something. I think we both agree we need more info from the OP to say anything useful. 16B alignment is the largest you'll usually see, though, because AVX requires efficient support for unaligned memory. So spilling ymm regs doesn't suffer too badly or require extra `mov`s if done with unaligned loads/stores, unlike for non-VEX SSE. – Peter Cordes Mar 06 '16 at 06:29
  • [Why does GCC allocate more space than necessary on the stack?](https://stackoverflow.com/a/63010237) demonstrates GCC over-allocating stack space that it doesn't need for alignment. – Peter Cordes Jul 21 '20 at 08:17