Memory corruption calling C++ dll from VB.NET

Question

I'm attempting to call a function in a C++ dll from vb.net, but am running into the following error:

Managed Debugging Assistant 'PInvokeStackImbalance' has detected a problem in 'D:...\calling_project_in_VB.vshost.exe'.

Additional information: A call to PInvoke function 'calling_project_in_VB! calling_project_in_VB.Module1::add' has unbalanced the stack. This is likely because the managed PInvoke signature does not match the unmanaged target signature. Check that the calling convention and parameters of the PInvoke signature match the target unmanaged signature.

I'm hopeful someone might be versed enough in this stuff to spot where I'm going wrong? I would appreciate any suggestions, so far the googles has been unproductive in helping me identify what the issue is. Here is the small reproducible example I'm attempting to run:

The C++ is set up as such:

called_c.h:

extern "C"  __declspec( dllexport ) 
                   int add(int* a, int* b);

called_c.cpp:

#include "stdafx.h"
#include "called_c.h"
#include <string>

using namespace std;
// using namespace System;

int add(int* a, int* b)
{
    int Aa = *a;
    int Bb = *b;

    return Aa + Bb;
}

And here is the VB function attempting to call the C (where "..." is the path on my machine):

Imports System.Runtime.InteropServices
Imports System

Module Module1

    'Public Class called_c
      <DllImport("D:\...\called_c.dll", EntryPoint:="add", ExactSpelling:=False)>
    Public Function add(ByRef a As Int32, ByRef b As Int32) As Int32
    End Function

    'End Class

    Sub Main()

        Dim val1 As Int32
        Dim val2 As Int32
        Dim answer As Int32

        val1 = 3
        val2 = 4

        answer = add(val1, val2)

        MsgBox(answer)

    End Sub

End Module

Why on earth would you want to pass int pointers for just adding them (you *do not* need these by reference anyway). — crashmstr, Mar 08 '17 at 20:47
@crashmstr this is just a small reproducible example that I hope to get working as a precursor to a more complicated problem where 0-filled vectors will be passed into the c code, filled, and returned to the VB. If you have a small working example, I'd be very much obliged if you could share it. — user2256085, Mar 08 '17 at 22:02
I don't know if this will help, but you could try adding the [`<[In]()>` attribute](https://msdn.microsoft.com/en-us/library/system.runtime.interopservices.inattribute(v=vs.110).aspx) to the `ByRef` arguments to prevent the Marshaler from copying the value back. — TnTinMn, Mar 08 '17 at 22:43
@TnTinMn `CallingConvention:=CallingConvention.Cdecl` worked! On to the next modification of my small example before applying to "real-world" problem. — user2256085, Mar 09 '17 at 02:54

score 1 · Answer 1 · answered Mar 08 '17 at 21:27

1

Pretty much anything that is a pointer in C++ equals to IntPtr in the .NET world, thus your parameters should be of that type. However, as crashmstr says, why would you even need them to be pointers in the first place if you're only adding two numbers together?

<DllImport("D:\...\called_c.dll", EntryPoint:="add", ExactSpelling:=False)>
Public Function add(ByVal a As IntPtr, ByVal b As IntPtr) As Integer
End Function

Also note that the parameters should be passed as ByVal because the C++ function currently doesn't include a parameter that is expected to be passed by reference.

Usage example:

Dim Result As Integer = add(New IntPtr(3), New IntPtr(6))
MessageBox.Show(Result) 'Should display "9".

answered Mar 08 '17 at 21:27

Visual Vincent

18,045
5
28
75

adopting your suggestions changed the error message, so that seems like progress. Instead of getting the PInvoke error cited in my original question, I'm now getting, "An unhandled exception of type 'System.AccessViolationException' occurred in calling_project_in_VB.exe Additional information: Attempted to read or write protected memory. This is often an indication that other memory is corrupt." – user2256085 Mar 08 '17 at 22:06
@user2256085 : That's usually what you get when something goes wrong in the C++ code. The parameters should be of type `IntPtr`, so I rather think you're missing something in the C++ part. Try adding the `__stdcall` calling convention before/after `__declspec( dllexport )`. – Visual Vincent Mar 09 '17 at 00:10

score 1 · Accepted Answer · edited Mar 09 '17 at 00:15

1

The default calling convention for the DllImportAttribte is listed as:

The default value for the CallingConvention field is Winapi, which in turn defaults to StdCall convention.

However the default for C++ programs is __cdecl

Your C++ code appears to use this default value by not specifying an alternative, so the VB signature should be:

<DllImport("D:\...\called_c.dll", EntryPoint:="add", ExactSpelling:=False, CallingConvention:=CallingConvention.Cdecl)>
Public Function add(ByRef a As Int32, ByRef b As Int32) As Int32
End Function

Alternatively, you may want to add the InAttribute to the arguments to prevent any copying back of the values by the interop marshaler.

<DllImport("D:\...\called_c.dll", EntryPoint:="add", ExactSpelling:=False, CallingConvention:=CallingConvention.Cdecl)>
Public Function add(<[In]()> ByRef a As Int32, <[In]()> ByRef b As Int32) As Int32
End Function

edited Mar 09 '17 at 00:15

Visual Vincent

18,045
5
28
75

answered Mar 09 '17 at 00:08

TnTinMn

11,522
3
18
39

I just suggested he'd use StdCall, though both will work. :) – Visual Vincent Mar 09 '17 at 00:16
@VisualVincent, I'm curious as what you changed in the edit? My eyes are tired, but all I see is that the code blocks where deleted and then put back in. – TnTinMn Mar 09 '17 at 00:27
It's an invisible code tag. I added `` above your first code block. It changes so that all your code blocks use VB.NET's syntax highlighting. – Visual Vincent Mar 09 '17 at 07:37
@VisualVincent, thank you for information. It seems like I'm not the only one that can not detect those changes. This topic just popped up on the [Hot Meta Links sidebar](https://meta.stackoverflow.com/questions/345097/reviewer-thinks-there-is-no-change-in-documentation?cb=1). – TnTinMn Mar 09 '17 at 15:01
Heh. It is displayed in the `side-by-side markdown` mode. :) – Visual Vincent Mar 09 '17 at 15:37

Memory corruption calling C++ dll from VB.NET

2 Answers2