As execution of a function is completed, and instructions and local variables are removed from the stack, how is the return value stored in memory for the process' main program to use?
Asked
Active
Viewed 57 times
2 Answers
1
How parameters are passed in, and values returned from, an executed function is known as the Calling Convention.
Ignoring runtime environments (Java and .NET, I'm looking at you) and scripted languages (any of them) and concetrating purely on native code on x86, there's a lot of them. You may have come across them if you've ever heard the term cdecl or stdcall amongst others.
Typically return values will be returned in registers. The cdecl convention, for example, returns data either in EAX (for integers and pointers) or ST0 (for floating-point values).
But the calling convention defines more than just the return format. It also defines how arguments are passed on (stack, or register and left to right or right to left) and who is responsible for cleaning the stack up (i.e., the caller or the callee). cdecl for example is an example of a convetion where the caller must clean the stack up, whilst stdcall the callee keeps the stack tidy.
Other conventions include fastcall, pascal and syscall, amongst others. Wikipedia has a good breakdown on them all, as does Microsoft's MSDN notes. You may also want to look at the SO question "stdcall and cdecl" which goes into cdecl and stdcall in detail.
