-1

I'm trying to understand more about stack pointer's behavior through function calls, and I'm not sure I understand what happens when we call and return from a function. Assuming I have this main program:

int main()
{
    demo();
    return 0;
}

And demo is defined like this:

void demo()
{

}

I'm using VS2019, and when I debug I inspect the following SP values through time with respect to the assembly code (example for values for debug session):

  1. Before entering demo -

EAX = 00DDD006 EBX = 00608000 ECX = 00DDD006 EDX = 00000001 ESI = 00DD1023 EDI = 008FFBA8 EIP = 00DD1D4F ESP = 008FFAB8 EBP = 008FFBA8 EFL = 00000246

demo();
00DD1D4F call        _bar (0DD1410h) 
  1. Step into call -
    EAX = 00DDD006 EBX = 00608000 ECX = 00DDD006 EDX = 00000001 ESI = 00DD1023 EDI = 008FFBA8 EIP = 00DD1410 ESP = 008FFAB4 EBP = 008FFBA8 EFL = 00000246

and in assembly the position is:

00DD1410  jmp         demo (0DD1A30h)  
  1. One step into jmp function - EAX = 00DDD006 EBX = 00608000 ECX = 00DDD006 EDX = 00000001 ESI = 00DD1023 EDI = 008FFBA8 EIP = 00DD1A30 ESP = 008FFAB4 EBP = 008FFBA8 EFL = 00000246

The function flow in assembly:

void demo()
{
(1) 00DD1A30 push        ebp  
(2) 00DD1A31 mov         ebp,esp  
(3) 00DD1A33 sub         esp,0C0h  
(4) 00DD1A39 push        ebx  
(5) 00DD1A3A push        esi  
(6) 00DD1A3B push        edi  
(7) 00DD1A3C lea         edi,[ebp-0C0h]  
(8) 00DD1A42 mov         ecx,30h  
(9) 00DD1A47 mov         eax,0CCCCCCCCh  
(10)00DD1A4C  rep stos    dword ptr es:[edi]  
(11)00DD1A4E  mov         ecx,offset _2D317A6C_scratch_pad@c (0DDD00Ch)  
(12)00DD1A53  call        @__CheckForDebuggerJustMyCode@4 (0DD134Dh)  

}
  1. Step into (1), changes - EAX = 00DDD006 EBX = 00608000 ECX = 00DDD006 EDX = 00000001 ESI = 00DD1023 EDI = 008FFBA8 EIP = 00DD1A31 ESP = 008FFAB0 EBP = 008FFBA8 EFL = 00000246
  2. Step into (2), changes -
    AX = 00DDD006 EBX = 00608000 ECX = 00DDD006 EDX = 00000001 ESI = 00DD1023 EDI = 008FFBA8 EIP = 00DD1A33 ESP = 008FFAB0 EBP = 008FFAB0 EFL = 00000246
  3. Step into (3), changes - EAX = 00DDD006 EBX = 00608000 ECX = 00DDD006 EDX = 00000001 ESI = 00DD1023 EDI = 008FFBA8 EIP = 00DD1A39 ESP = 008FF9F0 EBP = 008FFAB0 EFL = 00000206
  4. Step into (4), changes - EAX = 00DDD006 EBX = 00608000 ECX = 00DDD006 EDX = 00000001 ESI = 00DD1023 EDI = 008FFBA8 EIP = 00DD1A3A ESP = 008FF9EC EBP = 008FFAB0 EFL = 00000206
  5. Step into (5), changes - EAX = 00DDD006 EBX = 00608000 ECX = 00DDD006 EDX = 00000001 ESI = 00DD1023 EDI = 008FFBA8 EIP = 00DD1A3B ESP = 008FF9E8 EBP = 008FFAB0 EFL = 00000206
  6. Step into (6), changes - EAX = 00DDD006 EBX = 00608000 ECX = 00DDD006 EDX = 00000001 ESI = 00DD1023 EDI = 008FFBA8 EIP = 00DD1A3C ESP = 008FF9E4 EBP = 008FFAB0 EFL = 00000206
  7. Step into (7), changes - EAX = 00DDD006 EBX = 00608000 ECX = 00DDD006 EDX = 00000001 ESI = 00DD1023 EDI = 008FF9F0 EIP = 00DD1A42 ESP = 008FF9E4 EBP = 008FFAB0 EFL = 00000206
  8. Step into (8), changes - EAX = 00DDD006 EBX = 00608000 ECX = 00000030 EDX = 00000001 ESI = 00DD1023 EDI = 008FF9F0 EIP = 00DD1A47 ESP = 008FF9E4 EBP = 008FFAB0 EFL = 00000206
  9. Step into (9), changes - EAX = CCCCCCCC EBX = 00608000 ECX = 00000030 EDX = 00000001 ESI = 00DD1023 EDI = 008FF9F0 EIP = 00DD1A4C ESP = 008FF9E4 EBP = 008FFAB0 EFL = 00000206
  10. Step OVER into (10), changes- EAX = CCCCCCCC EBX = 00608000 ECX = 00000000 EDX = 00000001 ESI = 00DD1023 EDI = 008FFAB0 EIP = 00DD1A4E ESP = 008FF9E4 EBP = 008FFAB0 EFL = 00000206
  11. Step over (11), changes- EAX = CCCCCCCC EBX = 00608000 ECX = 00DDD00C EDX = 00000001 ESI = 00DD1023 EDI = 008FFAB0 EIP = 00DD1A53 ESP = 008FF9E4 EBP = 008FFAB0 EFL = 00000206
  12. Step into (12), changes- EAX = CCCCCCCC EBX = 00608000 ECX = 00DDD00C EDX = 00000001 ESI = 00DD1023 EDI = 008FFAB0 EIP = 00DD134D ESP = 008FF9E0 EBP = 008FFAB0 EFL = 00000206

My questions are:

  1. Before entering demo, ESP = 008FFAB8, and then ESP changed - ESP = 008FFAB4. What inserted in those 4 bytes that caused the stack pointer to increase (downwards)?
  2. Is the EBP is essentially the 'Frame Pointer'? and in the 4 bytes underneath it we can assume the return address resides? and then the function's parameters?
  3. Is the diff EBP - ESP results in the memory allocated for the locals of the functions?
  4. Is every time we push something inside the demo scope (as done in (1)) then the stack pointer will increase?
  5. Will appreciate explanation on what exactly happens in the stack in steps (1)-(9).
Mark Rotteveel
  • 100,966
  • 191
  • 140
  • 197
IntToThe
  • 59
  • 6
  • You can also step into assembly code with VS2019. Use the Debug>Windows>Disassembly command. – Jabberwocky Jul 26 '21 at 07:46
  • I suspect the context for this question is [this previous question](https://stackoverflow.com/questions/68522620). – Steve Summit Jul 26 '21 at 11:22
  • It is unclear at what position in your code you are after "2. Step into `demo`, right in the beginning" and "3. One step into `demo` function." and what exactly means "4. Returning from `demo` into `main`". Step 4 might be in fact two steps for releasing the memory for the local variable and returning to the calling function. (The implementation of function calls depends highly on the processor architecture.) – Bodo Jul 26 '21 at 11:40
  • I've detailed my question, can it be re-opened? – IntToThe Jul 27 '21 at 07:26

3 Answers3

3

This is the compiler generated coode for the demo function:

void demo()
{
00331E10  push        ebp  
00331E11  mov         ebp,esp  
00331E13  sub         esp,0C0h    // <<<< you are probably refering to this
00331E19  push        ebx  
00331E1A  push        esi  
00331E1B  push        edi  
00331E1C  mov         edi,ebp  
00331E1E  xor         ecx,ecx  
00331E20  mov         eax,0CCCCCCCCh  
00331E25  rep stos    dword ptr es:[edi]  
00331E27  mov         ecx,offset _4C554807_foo@c (033C000h)  
00331E2C  call        @__CheckForDebuggerJustMyCode@4 (033130Ch)  
}

This seems a lot for a function that does nothing. All that code is only added in debug versions at it's purpose is for the debug runtime to be able to detect local buffer overflows:

Let's have a look at this small program, where we write beyond the end of the the local buffer test:

void demo()
{
  char test[20];

  for (int i = 0; i < 30; i++)
    test[i] = 0;
}

int main()
{
  demo();
}

This is the generated assembly code for demo (comments are mine):

void demo()
{
003743C0  push        ebp  
003743C1  mov         ebp,esp  
003743C3  sub         esp,0F4h  
003743C9  push        ebx  
003743CA  push        esi  
003743CB  push        edi  
003743CC  lea         edi,[ebp-34h]  
003743CF  mov         ecx,0Dh  
003743D4  mov         eax,0CCCCCCCCh  
003743D9  rep stos    dword ptr es:[edi]  
003743DB  mov         ecx,offset _4C554807_foo@c (037C000h)  
003743E0  call        @__CheckForDebuggerJustMyCode@4 (037130Ch)  
  char test[20];

  for (int i = 0; i < 30; i++)
003743E5  mov         dword ptr [ebp-24h],0  
003743EC  jmp         __$EncStackInitStart+2Bh (03743F7h)  

// start of for loop
003743EE  mov         eax,dword ptr [ebp-24h]         // [ebp-24h] is i
003743F1  add         eax,1                           // i++
003743F4  mov         dword ptr [ebp-24h],eax  

003743F7  cmp         dword ptr [ebp-24h],1Eh        // i > 30 (0x1e)
003743FB  jge         _scanf+3h (0374423h)           //  yes -> go to end of for loop

// check for local local buffer overflow
003743FD  mov         eax,dword ptr [ebp-24h]  
00374400  mov         dword ptr [ebp-0F0h],eax  
00374406  cmp         dword ptr [ebp-0F0h],14h  
0037440D  jae         __$EncStackInitStart+45h (0374411h)  
0037440F  jmp         __$EncStackInitStart+4Ah (0374416h)  
00374411  call        ___report_rangecheckfailure (0371046h)  
00374416  mov         ecx,dword ptr [ebp-0F0h]  
// end check for local local buffer overflow

0037441C  mov         byte ptr test[ecx],0          // test[i] = 0
00374421  jmp         __$EncStackInitStart+22h (03743EEh)  
}
Jabberwocky
  • 48,281
  • 17
  • 65
  • 115
2

The function call instruction causes the CPU to push the return address to the stack, which is 4 bytes in this case. The function itself can also allocate more stack space.

In Visual C++, what you are seeing is that the { line represents the stack setup code in the function (the prologue). When the next instruction is the {, the call has executed, but the function prologue has not. So 4 bytes have been taken to store the return address, but any additional bytes the function wants to use have not been allocated. When you step over {, that is the function prologue which sets up the rest of the stack frame for the function.

Jabberwocky's answer provides the assembly code for the function, but doesn't really explain why you see the stack pointer decremented in two parts.

user253751
  • 57,427
  • 7
  • 48
  • 90
1

The stack pointer is of course an implementation detail. There's no defined way in C to see what the stack pointer is, or to know how it works -- in fact there's no guarantee that there even is a conventional stack or a stack pointer.

There are usually two (or more) active pointers into the stack. There's usually a "stack pointer" which keeps track of how much data has been pushed to the stack, and a "frame pointer" which always points to the base of the current function's stack frame.

Normally the stack pointer increases by sizeof(int) each time a word is pushed to the stack. For example, if you called f(1, 2, 3), you might see the stack pointer change by 3*sizeof(int) as the three arguments are pushed, and before f gets called. (Actually, it's more complicated than that, because these days some arguments are typically passed to function in registers, not on the stack at all.)

Normally the stack pointer grows by quite a bit when a function is called, because a whole new stack frame has to be created. Among other things, the old stack pointer and the return address have to be saved, and maybe a link pointer to the calling function's stack frame.

The stack pointer may change for other reasons, too. Local variables are stored on the stack, so when a new function is called, it may adjust the stack pointer to leave room for them. It's also possible for your function to do things which may cause extra memory to be allocated on the stack -- and therefore the stack pointer to be further adjusted -- midway through a function. If you declare a "variable-length array" (VLA) by writing

int a[n];

where n is a variable not known until run time, the stack pointer will be adjusted by n*sizeof(int). More or less the same thing happens if you call the old alloca function. If you declare additional local variables in the inner block of a loop or conditional statement, the stack pointer may be further adjusted when that block is entered.

I mentioned the "frame pointer", which is typically only changed once when a function is called, and then stays the same for as long as that function is active. The frame pointer is what's used to access local variables -- each local variable is stored at some known, fixed offset from the frame pointer.

When a function returns, the stack and frame pointers and the program counter are all restored to what they were just before the function got called, based on the values saved in the stack frame.

Steve Summit
  • 45,437
  • 7
  • 70
  • 103