7

I've spent all day researching this, and I'm none the wiser:

I have a C# DLL which PInvokes a method in a C++ DLL. I have had no problems doing this when compiling in Debug mode, but when compiling in Release mode I get an AccessViolationException. Googling this problem tells me that it's probably an issue with incompliant calling conventions. The code now looks like this in C#:

[return: MarshalAs(UnmanagedType.U1)]
[DllImport("Native.dll", CallingConvention = CallingConvention.Cdecl)]
internal static extern Boolean AMethod(Int32 mode, byte frame);

and in C++:

extern "C" {
     DLL_EXPORT bool AMethod(int mode, BYTE frame)
     {
      ...
     }
}

I've set the C++ project to compile with the __cdecl calling convention in VS2010, but I still get the AccessViolationException and I have no idea what more I can do. I should note that my C++ DLL uses third-party DLLs, and I do not know what calling convention they use.

Any help would be appreciated!

Oh, and I don't get the exception on my development machine, only on my target system.

PNielsen
  • 123
  • 5
  • 4
    Did you set up your C++ project to use the `__cdecl` convention in both Debug and Release builds? – Frédéric Hamidi May 09 '11 at 14:41
  • Does your DLL uses ATL ? If true, then try setting the Use of ATL to "Static Link to ATL" – João Augusto May 09 '11 at 15:18
  • 1
    Have you tried switching to __stdcall? I've never had good luck with the Cdecl calling convention with pinvoke, I always use __stdcall – Justin May 09 '11 at 15:34
  • Frederic: Yes, I did. Joao: The DLL uses MFC - but I'll give it a try anyway. Justin: I have, and it didn't seem to make a difference - I'm wondering if that's because the C++ DLL is dependent on other DLLs which are use cdecl? – PNielsen May 09 '11 at 18:30
  • 1
    @PNielsen, calling conventions are not viral, so I don't think external dependencies are the culprit here. I'm more worried about the `DLL_EXPORT` macro, what exactly does it expand to? – Frédéric Hamidi May 09 '11 at 20:45
  • Oh, sorry, I should have clarified that: `#define DLL_EXPORT __declspec(dllexport)` But thanks, the fact that calling conventions aren't viral helps a lot - that leaves me the option of experimenting more with __stdcall. – PNielsen May 09 '11 at 20:50
  • 1
    Okay, so it turns out the exception was thrown for another reason (which I still haven't discovered), unrelated to the calling convention specified. Thanks for all your help, at least I learnt a few things about calling convention on the way, particularly from this website (in case anyone else has this problem): http://www.unixwiz.net/techtips/win32-callconv.html – PNielsen May 10 '11 at 12:57

1 Answers1

0

Try with this reordering of statements :

[DllImport("Native.dll", CallingConvention = CallingConvention.Cdecl)]
[return: MarshalAs(UnmanagedType.U1)]
internal static extern Boolean AMethod(Int32 mode, byte frame);
lsalamon
  • 7,998
  • 6
  • 50
  • 63