C function pointer crashes program?

Question

Im trying to write a function that gets the address of MessageBoxA from a structure pointer that I pass to it and then uses that address to call MessageBoxA. The problem is that I try to call MessageBoxA 5 times with this method, but my program always crashes after 3 times and i have no idea why.

void code(struct ShellD * s)
{

    s->pMsgBox(NULL,"s","s",MB_OK);

    s->pMsgBox(NULL,"s","s",MB_OK);

    s->pMsgBox(NULL,"s","s",MB_OK);

    s->pMsgBox(NULL,"s","s",MB_OK);

    s->pMsgBox(NULL,"s","s",MB_OK);
}

int main(int argc, char **argv)
{

struct ShellD SD;

SD.pMsgBox = GetProcAddress(LoadLibraryA("user32.dll"),"MessageBoxA");

code(&SD);

}

What's your ShellD structure? In particular, what calling convention have you declared on the pMsgBox type? — Rup, Oct 23 '20 at 21:41
pMsgBox is defined as int (*pMsgBox)(HWND,LPCSTR,LPCSTR,UINT); i know that this is not the problem because the first 3 MessageBoxes appear without a problem. Edit: I have tried other functions too, but no matter what function i try to call it always crashes after 3 times — Sinnlosername, Oct 23 '20 at 21:42
Like @Rup said, you need to use the correct calling convention on your function pointer. The declaration of `pMsgBox` *is* the problem, it is causing a stack misalignment from each call that gets worse and worse the more you use it. — Ben Voigt, Oct 23 '20 at 21:48
that actually worked :) What difference does the WINAPI make? — Sinnlosername, Oct 23 '20 at 21:53
@Sinnlosername See [What does “WINAPI” in main function mean?](https://stackoverflow.com/questions/2348442/) — Remy Lebeau, Oct 24 '20 at 01:28

score 5 · Accepted Answer · edited Oct 24 '20 at 01:37

pMsgBox is defined as int (*pMsgBox)(HWND,LPCSTR,LPCSTR,UINT);

You've dropped the WINAPI from the declaration of MessageBoxA:

int
WINAPI // <--
MessageBoxA(
    __in_opt HWND hWnd,
    __in_opt LPCSTR lpText,
    __in_opt LPCSTR lpCaption,
    __in UINT uType);

pMsgBox should be like this instead:

int (WINAPI *pMsgBox)(HWND,LPCSTR,LPCSTR,UINT)

What's the difference?

The Windows API functions use the stdcall calling convention (also known as the Pascal calling convention). One of the characteristics of stdcall functions is that they clean up the stack space used by any arguments passed in.
By contrast, in the default cdecl calling convention, the calling code cleans up the stack space for any arguments it passes in.

By using the wrong calling convention in your declaration of pMsgBox, both your code() function and the MessageBoxA() function are moving the stack pointer to remove passed arguments, and after a few times you've probably drifted the stack pointer out of the current stack frame and are then overwriting other things on the stack e.g. the return function pointer. This is probably what's causing the crash.

C function pointer crashes program?

1 Answers1