Create a Windows shared library with DllMain function (MinGW + CMake)

Question

I apologise if this has already been asked, but I have been trying to find an answer for many weeks now.

I am not particularly competent when it comes to shared libraries, however, using CLion RC 1.0 (and its integrated CMake) with MinGW, I have been unable to create a functional Windows DLL with a DllMain function. I did manage to create a .dll file, but it quickly became apparent that the symbol did not export corrrectly.

Simply put, I would like to see identical or similar behaviour to what is obtained with Visual Studio's default Win32 DLL template.

CMakeLists.txt

cmake_minimum_required(VERSION 3.1)
project(DllMainTest)

set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -std=c++11")

set(SOURCE_FILES main.cpp)
add_library(DllMainTest SHARED ${SOURCE_FILES})

main.cpp

#include <windows.h>

extern "C" BOOL WINAPI DllMain(
    HINSTANCE hinstDLL,
    DWORD fdwReason,
    LPVOID lpvReserved
) {
switch(fdwReason) {
    case DLL_PROCESS_ATTACH:
        MessageBox(NULL, "It works!", "Status", MB_OK);
    case DLL_PROCESS_DETACH:
    case DLL_THREAD_ATTACH:
    case DLL_THREAD_DETACH:
    default:
        break;
    }
return TRUE;
}

EDIT: The intention for this DLL is to have it injected into a 32-bit process and for it to display a MessageBox on load.

show us your CMakeLists.txt and complete main function please. — user1283078, Apr 07 '15 at 12:05
Calling a user32.dll function (`MessageBox`) from `DllMain` is almost guaranteed to result in a deadlock, sooner or later. See [Dynamic-Link Library Best Practices](https://msdn.microsoft.com/en-us/library/windows/desktop/dn633971.aspx#general_best_practices) for details. — IInspectable, Apr 07 '15 at 13:34
Also note, the `extern "C"` storage class specifier instructs the compiler to use C name mangling (when compiling as C++). It does not result in an entry in the export table. Use [__declspec(dllexport)](https://msdn.microsoft.com/en-us/library/a90k134d.aspx) to export the symbol from the DLL. Failure to use `break` statements probably means, that you shouldn't be injecting your code into another process to begin with. — IInspectable, Apr 07 '15 at 13:51
@IInspectable I understand the loader-locking idea (this is just a sample/PoC), but thanks for reminding me, and I'll be sure to create a thread. Thanks for the `__declspec(dllexport)` idea, but I'll have to contest on the `break` comment. Shouldn't execution fall through to the default case and break there ([Fallthrough](http://en.wikipedia.org/wiki/Switch_statement#Fallthrough))? — Benjamin Crawford, Apr 07 '15 at 14:01
The `default` label is in no way special. While common to have it as the final label, there is no requirement to do so. It could be the first label, or missing altogether. While the language allows to fall through labels, it's a good candidate to introduce subtle, hard to find bugs. Most static code analyzers will object to fallthroughs. — IInspectable, Apr 07 '15 at 14:10
@BenjaminCrawford By the way, I think you want to put a `break;` after your MessageBox call — Antonio, Apr 07 '15 at 21:13
@Antonio If you read the three comments above yours, you'll find the discussion about break statements. In this case, I really don't see the need for one. Sure, if I add code to other cases then sure (in fact I originally had break statements before posting, but removed them for ease of reading). Adding a break statement in this case will have no effect. — Benjamin Crawford, Apr 08 '15 at 10:01
@BenjaminCrawford Removing that `break` does not make the reading easier, rather the contrary. — Antonio, Apr 08 '15 at 11:13
@Antonio The various doctrines possessed by the global C++ programming community makes convention ambiguous, trivial at best. Let's not intrude on a space polluted with contrasting passions and focus on the point at hand, unless this apparently illicit practise is the cause of such issues. — Benjamin Crawford, Apr 09 '15 at 11:39
@IInspectable use of `__declspec(dllexport)` is not mandatory with MinGW; all symbols will be exported, by default, if **none** are so qualified. However, as soon as just one is so qualified, in any linked image, then it does become mandatory for all which are to be exported. — Keith Marshall, Apr 13 '15 at 05:35

score 1 · Answer 1 · answered Apr 07 '15 at 12:30

1

check out the add_library documentation: http://www.cmake.org/cmake/help/v3.0/command/add_library.html

"STATIC libraries are archives of object files for use when linking other targets. SHARED libraries are linked dynamically and loaded at runtime. MODULE libraries are plugins that are not linked into other targets but may be loaded dynamically at runtime using dlopen-like functionality"

add_library(DllMainTest MODULE ${SOURCE_FILES})

should export the DllMain as intended.

answered Apr 07 '15 at 12:30

user1283078

2,006
17
17

Thanks for the answer but, while it definitely looked promising (and does apply to my use case), DLL injection is still failing. – Benjamin Crawford Apr 07 '15 at 12:44
hm. that works fine for me when creating dlls for injection. Maybe you should skip injection for now and create a little exe that tries to load the library. Check the return value of LoadLibrary and get the error code with GetLastError. See the msdn article on LoadLibrary. Just a shot in the dark: You want to inject into a 32bit process. You have a 64bit version of mingw installed which would by default create a 64bit dll, which would ofc fail to be loaded into a 32bit target. – user1283078 Apr 07 '15 at 13:44
@user1283078 I believe to build a 32 bit version of the DLL you can use this command in the CMakeLists.txt file: set_target_properties(DllMainTest PROPERTIES COMPILE_FLAGS "-m32" LINK_FLAGS "-m32") – MikeyE Oct 14 '16 at 07:19

Create a Windows shared library with DllMain function (MinGW + CMake)

1 Answers1

Linked