Dynamic memory allocation in native dll

Question

I have a native (unmanaged) .dll written in C++ that is to be called from a managed process (a C# program). When debugging the dll the problem I have has shown to be that when I create an object in the dll with the new keyword I get a System Access Violation Exception. This only shows up when calling the dll from a managed process, not when I am calling it from another native program.

The code is something similar to this:

// Native.dll file
MyClass myInstance; // global variable (and does need to be so)
__declspec(dllexport) uint8_t _stdcall NativeFunction(){
    myInstance = new MyClass(); // <-- this causes Access Violation Exception
}

and the C# code:

using System.Runtime.Interopservices;
// Loading the dll
[DllImport("Native.dll",CallingConvention = CallingConvention.StdCall)]
private extern static byte NativeFunction();

class TestClass{
   byte returnVal = NativeFunction(); //<-- exception in managed context
}

I know this has something to do with the native process trying to allocate memory outside allowed memory-space. It only happens when memory is allocated with new (at least in this project) which I unfortunately do need to use. My question is: Does anyone know why this is causing the exception and how to avoid it?

Answer 1

new MyClass is most likely going to call ::operator new, the global operator, unless you have provided MyClass::operator new. And if you haven't provided ::operator new yourself, you should be getting the ::operator new from your compiler (likely Visual Studio).

This ::operator new implementation will probably forward to HeapAlloc. And guess what? That's the same Win32 function which .Net will call as well. There's not much magic involved here; that's how Windows assigns pages of memory to your virtual address space. And when you use those pages, Windows will assign RAM.

Now the thing here is that you don't need to do anything special for this. In fact, doing anything special is how you would break operator new. And since you broke it, you''re going to have to figure that out. There is not much magic code going on here. Use a debug build, so you will have a clear stack dump (no inlining). Can you backtrack to HeapAlloc?

Check also the content of the Access Violation Exception. The error code will be C0000005. But what type of exception is it? Read or write? On what type of address? Code or data?