6

I am using a third party C library with the following opaque handle:

typedef struct _VendorHandle *VendorHandle;

Here is the vendor's C example of how to load the handle:

VendorHandle handle;
int err;
err = vendorLoadFile(&handle, "something.bin");

I am trying to call this method in C# with PInvoke using the following declaration:

[DllImport("VendorLib.dll")]
static extern int vendorLoadFile(IntPtr handle, string path);

Then I added the following code to use the declaration:

IntPtr handle = new IntPtr();
int code = vendorLoadFile(handle, path);

When I run it I'm getting the following error:

Attempted to read or write protected memory. This is often an indication that other memory is corrupt.

I know that the DLL is fine and PInvoke is working because I am executing their vendorVersion() method so it has to be something else I'm doing wrong.

Brent Matzelle
  • 4,073
  • 3
  • 28
  • 27

1 Answers1

11

That function takes a pointer to an opaque handle, so that it can write the handle into the memory pointed to by the pointer.
In C# terms, that's an out IntPtr:

[DllImport("VendorLib.dll")]
static extern int vendorLoadFile(out IntPtr handle, string path);

IntPtr handle;
int code = vendorLoadFile(out handle, path);
SLaks
  • 868,454
  • 176
  • 1,908
  • 1,964
  • Thank you. I'm getting another error now: "A call to PInvoke function '[snip] vendorLoadFile' has unbalanced the stack. This is likely because the managed PInvoke signature does not match the unmanaged target signature. Check that the calling convention and parameters of the PInvoke signature match the target unmanaged signature." – Brent Matzelle Nov 27 '13 at 21:57
  • So did you do what the message told you? – David Heffernan Nov 27 '13 at 22:38
  • 2
    Okay, I got past that odd error. I just needed to add `CallingConvention = CallingConvention.Cdecl` to the DllImport as per [this StackOverflow answer](http://stackoverflow.com/a/6768223/330110). Thanks! – Brent Matzelle Nov 27 '13 at 22:42