How to resolve access violation writing location when calling dll method

Question

I'm using GetProcAddress to gain access to a standard Isapi Filter DLL method - the GetFilterVersion method which takes a pointer to a HTTP_FILTER_VERSION structure.

https://msdn.microsoft.com/en-us/library/ms525822(v=vs.90).aspx

https://msdn.microsoft.com/en-us/library/ms525465(v=vs.90).aspx

I've tested the code against a working Isapi filter that I've written and it works fine. I debug the code against an Isapi filter from a vendor (I don't have access to the source code or anything beyond the dll itself) and I get the exception, "access violation writing location". What could be the issue? (Both Isapi filters work in IIS.)

//Attempted to define function ptr several ways
typedef BOOL(__cdecl * TRIRIGAISAPIV)(PHTTP_FILTER_VERSION);
//typedef BOOL( * TRIRIGAISAPIV)(PHTTP_FILTER_VERSION);
//typedef BOOL(WINAPI * TRIRIGAISAPIV)(PHTTP_FILTER_VERSION);

void arbitraryMethod()
{
   HINSTANCE hDLL;               // Handle to DLL
   TRIRIGAISAPIV lpfnDllFunc2;    // Function pointer

   DWORD lastError;
   BOOL  uReturnVal2;

   hDLL = LoadLibrary(L"iisWASPlugin_http.dll");  //vendor's dll
   //hDLL = LoadLibrary(L"KLTWebIsapi.dll   //my dll

   if (hDLL != NULL)
   {
       lpfnDllFunc2 = (TRIRIGAISAPIV)GetProcAddress(hDLL, "GetFilterVersion");

       if (!lpfnDllFunc2)
       {
           lastError = GetLastError();
           // handle the error
           FreeLibrary(hDLL);
           //return 1;
       }
       else
       {            
           HTTP_FILTER_VERSION pVer = { 6 };

           //Call the function via pointer; Works with my dll, fails with vendor's
           uReturnVal2 = lpfnDllFunc2(&pVer);

           //................  HELP!!!!!!!!!!!!!
       }
   }
}

`BOOL(__cdecl ` Are you sure that the calling convention for this function is `__cdecl`, and not `__stdcall`? Most API functions are `__stdcall`, with the most notable exception being `wsprintf` (since it takes a vararg parameter list). — PaulMcKenzie, Jan 20 '16 at 22:58
I've tried __stdcall, __cdecl , WINAPI - doesn't work with either.... — user3390835, Jan 20 '16 at 23:24
Stack Trace points to Unhandled exception at 0x000000007744F6C6 (ntdll.dll) RtlEnterCriticalSection() — user3390835, Jan 20 '16 at 23:42

score 2 · Answer 1 · edited May 23 '17 at 12:31

2

One issue that I see is that your function pointer declaration is incorrect.

According to the Microsoft documentation, GetFilterVersion is prototyped as:

BOOL WINAPI GetFilterVersion(PHTTP_FILTER_VERSION pVer);

The WINAPI is a Windows macro that is actually defined as __stdcall, thus you are declaring the function pointer incorrectly when you used __cdecl.

What does WINAPI mean?

Thus, your declaration should be:

typedef BOOL(__stdcall * TRIRIGAISAPIV)(PHTTP_FILTER_VERSION);

edited May 23 '17 at 12:31

Community

1
1

answered Jan 20 '16 at 23:14

PaulMcKenzie

34,698
4
24
45

I had previously tried various declarations - doesn't work. Stack Trace fails in ntdll.dll. – user3390835 Jan 20 '16 at 23:22

EmDroid · Answer 2 · 2016-01-21T00:19:09.717

It could be that there are actually some additional structure fields filled by the custom filter.

You can try to increase the size of the structure to see if that will work, like for example:

struct HTTP_FILTER_VERSION_EXTRA {
    HTTP_FILTER_VERSION v;
    char[1024] extra;
};

HTTP_FILTER_VERSION_EXTRA ver;
ver.v.dwServerFilterVersion = 6;
uReturnVal2 = lpfnDllFunc2(&ver.v);

It is sometimes the case with the WinAPI structures that they allow versioning, so adding fields is possible. If the function doesn't then check (or doesn't know) the actual structure version, it might try to use an extended one which might be different than the one supplied - if the size of the supplied struct is then lesser than the structure version the func tries to use, bad things can happen.

Also check if the DLL is 64-bit or 32-bit. You cannot use 64-bit DLL by 32-bit app and vice versa (but I expect that would already fail during the LoadLibrary call).

How to resolve access violation writing location when calling dll method

2 Answers2