15

In a dll project, the function is like this:

extern "C" __declspec(dllexport) void foo(const wchar_t* a, const wchar_t* b, const wchar_t* c)

In a different project, I will use foo function, but I declare foo function in header file with 

extern "C" __declspec(dllimport) void foo(const wchar_t* a, const wchar_t* b)

and I call it with only two parameters.

The result is success, I think it about __cdecl call, but I would like to know how and why this works.

Mark Rotteveel
  • 100,966
  • 191
  • 140
  • 197
Frank
  • 153
  • 6
  • Well it will depend what foo does with the missing parameters, they will have random values. If as in the question they are pointers if foo tries to use them your application will most likely crash. – Richard Critten Mar 31 '17 at 07:13
  • The call and return will leave the stack intact, because in cdecl the caller is to remove the parameters from the stack. Of course, if the missing parameters are accessed by the called function, they will have undefined values or cause undefined behavior. – Paul Ogilvie Mar 31 '17 at 07:15
  • 2
    @RichardCritten not _random values_ but _indeterminate values_. – Jabberwocky Mar 31 '17 at 07:18
  • @MichaelWalz Isn't that, for all terms and purposes, the same? I see: *random* might be easier to spell... – tofro Mar 31 '17 at 07:19
  • 3
    @tofro I think MichaelWalz's point is that the values aren't, for instance, randomly generated. They might actually be pretty predictable in practice (e.g., maybe the stack aligns such that the value is always the value of some local variable in the caller). But that wouldn't be a behavior that's specified. This can make a bigger difference when asking "what would happen if..." because just testing it might reliably work in one environment, but not in another. With genuinely random values, you'd expect to similar distributions of values in different environments. – Joshua Taylor Mar 31 '17 at 13:02
  • @JoshuaTaylor Does *Random Access Memory* imply the access is *randomly generated*? – tofro Mar 31 '17 at 13:15
  • 1
    @tofro No, but it means that a sequence of memory accesses that are randomly generated should perform about as well as a series of sequential addresses. I'm not trying to be (too) pedantic, but just trying to clarify what I expect was MichaelWalz's intent. ["Indeterminate Value"](http://stackoverflow.com/questions/13423673/what-is-indeterminate-value) is a technical term in the standard, with precise meaning, where as "random values" might not be. – Joshua Taylor Mar 31 '17 at 13:18
  • @tofro The accepted answer to [this question](http://stackoverflow.com/q/13423673/1281433) mentions (about uninitialized variables): "in most implementations of C, an uninitialized scope variable or the memory pointed to by the pointer returned by a call to malloc will contain whatever value happened to be stored at that address previously." A **randomly generated** would also be indeterminate, but not necessarily useful. An indeterminate value that, in practice, has predictable contents like memory contents, can be **very useful**, e.g., to an attacker (maybe it contains some secret data). – Joshua Taylor Mar 31 '17 at 13:22
  • @tofro The general difference between random and indeterminate is: If it's random it's random, "randomness" is an actual quality of something. If it's indeterminate then we don't even *know* if it's random. You wouldn't want to blindly use an *indeterminate* value as a source of *randomness*, for example, because it might not be random at all. In this case, though, like you say, for all intents and purposes it's the same in that it has no impact on the end result that `c` is not useful, and it doesn't actually *matter* to us if it's random vs. indeterminate here. – Jason C Apr 02 '17 at 14:00

1 Answers1

32

32-bit

Default calling convention is __cdecl, which means the caller pushes parameters onto the stack right-to-left then cleans up the stack after the call returns.

So in your case, the caller:

  1. Pushes b
  2. Pushes a
  3. Pushes the return address
  4. Calls the function.

At this point the stack looks like this (assume 4 byte pointers for example, and remember the stack pointer travels backwards when you push things):

+-----+ <--- this is where esp is after pushing stuff
| ret | [esp]
+-----+
|  a  | [esp+4]
+-----+
|  b  | [esp+8]
+-----+ <--- this is where esp was before we started
| ??? | [esp+12 and beyond]
+-----+

Ok, great. Now the problem happens on the callee side. The callee is expecting parameters to be at certain locations on the stack, so:

  • a is assumed to be at [esp+4]
  • b is assumed to be at [esp+8]
  • c is assumed to be at [esp+12]

And this is where the issue is: We have no idea what's at [esp+12]. So the callee will see the correct values of a and b, but will interpret whatever unknown garbage happens to be at [esp+12] as c.

At that point it's pretty much undefined, and depends on what your function actually does with c.

After all this is over and the callee returns, assuming your program didn't crash, the caller will restore esp and the stack pointer will be back where it should be. So from the caller's POV everything is probably fine and the stack pointer ends up back where it's supposed to be, but the callee sees junk for c.

64-bit

The mechanics on 64-bit machines is different but the end result is roughly the same effect. Microsoft uses the following calling convention on 64-bit machines regardless of __cdecl or whatever (any convention you specify is ignored and all are treated identically):

  • First four integer or pointer arguments placed in registers rcx, rdx, r8, and r9, in that order, left-to-right.
  • First four floating-point arguments placed in registers xmm0, xmm1, xmm2, and xmm3, in that order, left-to-right.
  • Anything remaining is pushed to the stack, right-to-left.
  • The caller is responsible for restoring esp as well as restoring the values of all volatile registers after the call.

So in your case, the caller:

  1. Puts a in rcx.
  2. Puts b in rdx.
  3. Allocates an extra 32 bytes of "shadow space" on the stack (see that MS article).
  4. Pushes the return address.
  5. Calls the function.

But the callee is expecting:

  • a assumed to be in rcx (check!)
  • b assumed to be in rdx (check!)
  • c assumed to be in r8 (problem)

And so, as with the 32-bit case, the callee interprets whatever happened to be in r8 as c, and potential hijinks ensue, with the end effect depending on what the callee does with c. When it returns, assuming the program did not crash, the caller restores all volatile registers (rcx and rdx, and also generally includes r8 and friends) and restores esp.

Christopher Moore
  • 15,626
  • 10
  • 42
  • 52
Jason C
  • 38,729
  • 14
  • 126
  • 182
  • 6
    Note that [for 64 bit binaries](https://msdn.microsoft.com/en-us/library/ms235286.aspx), arguments will get passed in registers instead. Besides that, the effects are pretty much identical from what is described here - the unknown garbage is just read from a register instead of stack memory. – ComicSansMS Mar 31 '17 at 09:19
  • 1
    Maybe you should add a mention that callee cleaned calling conventions like stdcall on x86 will result in the stack becoming unbalanced, which pretty much guarantees a crash at the next return at least. – poizan42 Mar 31 '17 at 12:11
  • Also worth noting if the called code *never reads or writes `c`* (and assuming `__cdecl` etc) this particular code would "run fine". – Yakk - Adam Nevraumont Mar 31 '17 at 13:28
  • 1
    I haven't tried this in purely C++, but in my experience calling a C++ DLL from C#, Visual Studio is really good about giving you a message saying that the stack has become unbalanced, and self correcting. It does lead to major efficiency issues, however. – Cody Mar 31 '17 at 17:12