"Bad DLL calling convention" while using a C++ dll from Excel VBA - How to resolve?

Question

I need to create a C++ dll that have one function that returns a string to be used in an Excel (2010) VBA code.

I've read the following posts: Using C++ DLL in Excel VBA - and Microsoft's tutorials about C++ dll creation and VBA usage (Walkthrough: Create and use your own Dynamic Link Library (C++), Access DLLs in Excel) and followed the procedure.

Here is the code:

C++ dll (taken from Microsoft web page):

// MathLibrary.h - Contains declarations of math functions
#pragma once

#ifdef MATHLIBRARY_EXPORTS
#define MATHLIBRARY_API __declspec(dllexport)
#else
#define MATHLIBRARY_API __declspec(dllimport)
#endif

// The Fibonacci recurrence relation describes a sequence F
// where F(n) is { n = 0, a
//               { n = 1, b
//               { n > 1, F(n-2) + F(n-1)
// for some initial integral values a and b.
// If the sequence is initialized F(0) = 1, F(1) = 1,
// then this relation produces the well-known Fibonacci
// sequence: 1, 1, 2, 3, 5, 8, 13, 21, 34, ...

// Initialize a Fibonacci relation sequence
// such that F(0) = a, F(1) = b.
// This function must be called before any other function.
extern "C" MATHLIBRARY_API void fibonacci_init(
    const unsigned int a, const unsigned int b);

// Produce the next value in the sequence.
// Returns true on success and updates current value and index;
// false on overflow, leaves current value and index unchanged.
extern "C" MATHLIBRARY_API bool fibonacci_next();

// Get the current value in the sequence.
extern "C" MATHLIBRARY_API unsigned int fibonacci_current();

// Get the position of the current value in the sequence.
extern "C" MATHLIBRARY_API unsigned fibonacci_index();

the VBA code (Mine, following Microsoft documentation):

Public Declare Sub fibonacci_init Lib "C:\development\MathLibrary\Release\MathLibrary.dll" (ByVal a As Integer, ByVal a As Integer)
Public Declare Function fibonacci_next Lib "C:\development\MathLibrary\Release\MathLibrary.dll" () As Boolean
Public Declare Function fibonacci_current Lib "C:\development\MathLibrary\Release\MathLibrary.dll" () As Integer

Public Function run_dll()
    Dim b As Integer

    Call fibonacci_init(1, 1)
    b = fibonacci_current()
End Function

When I run the run_dll function in VBA i get an exception: "Bad DLL calling convention" on the Call fibonacci_init(1,1) line.

What is wrong here? I've declared the C++ function as extern "C" so I assumed calling convention are fixed...

UPDATE

More things i've tried...

I've created a new dll from scratch following the tips from the comments/answers:

Trial.cpp:

const char* str = "abcdefg";

extern "C" __declspec(dllexport) int size()
{
    return strlen(str);
}

extern "C" __declspec(dllexport) bool test(char* pReturn)
{
    int nSize = strlen(str);
    lstrcpynA(pReturn, str, nSize);

    return true;
}

with the following VBA:

Public Declare Function size Lib "C:\development\MathLibrary\Release\Trial.dll" () As Long
Public Declare Function test Lib "C:\development\MathLibrary\Release\Trial.dll" (ByVal p As Long) As Boolean

(1) Public Function run_dll()
(2)    Dim bb As Boolean
(3)    Dim sz As Integer
(4)    Dim s As String        
(5)    sz = size()       
(6)    s = Space(sz)        
(7)    bb = test(StrPtr(s))
(8) End Function

line 5 works fine - sz receives 7. But line 7 gives "Bad DLL calling convention".

Trying to declare the C++ function with WINAPI as one mentioned in a post i've read, gives: Can't find DLL entry point size in C:\development\MathLibrary\Release\Trial.dll.
Changing the test VBA declaration to

Public Declare Function test Lib "C:\development\MathLibrary\Release\Trial.dll" (ByRef p As String) As Boolean

and calling (line7) as bb = test(s) - causes Excel to crash

Changing the test VBA declaration to

Public Declare Function test Lib "C:\development\MathLibrary\Release\Trial.dll" (ByRef p As Long) As Boolean

and calling (line7) as bb = test(StrPtr(s)) - gives: "Bad DLL calling convention"

Seems nothing is working. Does someone have a working example for such setup?

Possible duplicate of [Debug a bad DLL calling convention error](https://stackoverflow.com/questions/574587/debug-a-bad-dll-calling-convention-error) — Alan Birtles, Sep 17 '18 at 06:39
@AlanBirtles: The post talks about MSAccess VBA - I don't follow the details there... I don't have a "module to VBA compiler" — NirMH, Sep 17 '18 at 06:57
@AlanBirtles: it talks about recompiling an Excel add-in.... not a C++ dll — NirMH, Sep 17 '18 at 07:03
Yes, make a change to your excel code to cause it to recompile, as it has 24 upvotes with the same error message as you I imagine there's a reasonable chance it will work — Alan Birtles, Sep 17 '18 at 07:05
@AlanBirtles: sorry, just tried - no change - still i get the error message. this is something else - might be the way i declare the C++ functions or the VBA "decoration" — NirMH, Sep 17 '18 at 07:12
@AlanBirtles The answers there don't address the current issue (a data type mismatch) — Erik A, Sep 17 '18 at 07:28
Please don't change the subject of your question halfway through. Passing strings to C++ DLLs is a distinct subject, with it's own set of problems. — Erik A, Sep 17 '18 at 17:56

Erik A · Answer 1 · 2018-09-17T07:37:39.527

1

The equivalent of a C++ Integer is a VBA long.

An Integer in VBA is a 2-byte data type, a Long in VBA is a 4-byte data type. Since the amount of bytes passed doesn't match the amount of bytes expected, you get a Bad DLL calling convention

Do note that VBA doesn't support unsigned types, thus your output gets interpreted as a signed 4-byte integer (long).

edited Sep 17 '18 at 07:37

answered Sep 17 '18 at 07:27

Erik A

31,639
12
42
67

Tried changing the VBA declaration to `(ByRef a As Long, ByRef b As Long)` and the C++ to `extern "C" __declspec(dllexport) void fibonacci_init(int a, int b);` - still same thing... – NirMH Sep 17 '18 at 08:07
`ByVal` not `ByRef`!! Why would you change that? – Erik A Sep 17 '18 at 08:12
Thanks for your help - but even with `ByVal` I still receive the error. Is it ok to ask you to post some code (simple and short) that is working? – NirMH Sep 17 '18 at 08:14
Unfortunately, I don't have access to a C++ compiler in my current workspace, so I can't compile DLLs. But afaik you should've just changed your VBA code. It's pretty usual in VBA to use signed types where you should actually use unsigned type. I think removing `Const` in your C++ code caused this not to work. I do have code to cast longs that should be unsigned but are signed to doubles to account for the larger values. – Erik A Sep 17 '18 at 08:20

score 0 · Answer 2 · answered Sep 17 '18 at 06:41

0

You will need to make it as a COM library. These can then be imported into the VBA Code once they are registered in the Windows registry.

answered Sep 17 '18 at 06:41

user2956314

174
4

1

I wanted to avoid registration on the running windows machine - i want to deploy the dll as part of my excel package - is there any option without COM and registration? – NirMH Sep 17 '18 at 06:43

"Bad DLL calling convention" while using a C++ dll from Excel VBA - How to resolve?

2 Answers2