linking dilemma (undefined reference) between MinGW and MSVC. MinGW fails MSVC works

Question

I'm trying to port an old C .dll library originally done with MSVC that uses BEA Tuxedo library to use MinGW.

I have encountered a situation where MSVC compiles and links one file but MinGW fails. The actual problem is in linking stage. There comes 'undefined reference' error.

Here's the minimal example to create a dll: (tpsetunsol_test.c)

#include <atmi.h>

void __stdcall msghandler(char *pszMessage, long lMessageLen, long lFlags)
{

}   

int Inittpsetunsol()
{           
    int ret = 0;

    tpsetunsol(msghandler);

    return ret;                     
}

This compiles without errors:

gcc -Wall -fexceptions -g -O2 -DWIN32 -DNDEBUG -D_WINDOWS -ID:/dev/tuxedo/include   -o   tpsetunsol_test.o -c tpsetunsol_test.c

Here comes the error:

dllwrap --export-all-symbols -LD:/dev/tuxedo/lib -k --output-lib test.lib --output-def test.def --enable-stdcall-fixup --add-stdcall-alias -o IAWS.dll tpsetunsol_test.o -lkernel32 -luser32 -lgdi32 -lwinspool -lcomdlg32 -ladvapi32 -lshell32 -lwtuxws32

C:\MinGW\bin\dllwrap.exe: no export definition file provided.
Creating one, but that may not be what you want
tpsetunsol_test.o: In function `Inittpsetunsol':
d:\dev\tpsetunsol_test.c:13: undefined reference to `tpsetunsol'
collect2.exe: error: ld returned 1 exit status
C:\MinGW\bin\dllwrap.exe: C:\MinGW\bin\gcc exited with status 1

function declaration in atmi.h:

extern void (_TMDLLENTRY * _TMDLLENTRY tpsetunsol _((void (_TMDLLENTRY *)(char _TM_FAR *, long, long)))) _((char _TM_FAR *, long, long));

#define _TMDLLENTRY __stdcall
#define _TM_FAR

Version:

$ gcc -v
Using built-in specs.
COLLECT_GCC=C:\MinGW\bin\gcc.exe
COLLECT_LTO_WRAPPER=c:/mingw/bin/../libexec/gcc/mingw32/4.7.2/lto-wrapper.exe
Target: mingw32
Configured with: ../gcc-4.7.2/configure --enable-    languages=c,c++,ada,fortran,objc,obj-c++ --disable-sjlj-exceptions --with-dwarf2 --enable-shared --enable-libgomp --disable-win32-registry --enable-libstdcxx-debug --disable-build-poststage1-with-cxx --enable-version-specific-runtime-libs     --build=ming
w32 --prefix=/mingw
Thread model: win32
gcc version 4.7.2 (GCC)

Edit: I found out using nm on object files created by MSVC and GCC that symbol tpsetunsol is different

Looking at _tpsetunsol symbol it is quite evident that MSVC and GCC produce different symbols.

GCC produces: U _tpsetunsol and MSVC: U _tpsetunsol@4

Edit: nm output after build with Haroogan's suggestion:

$ dllwrap --export-all-symbols -LD:/dev/tuxedo.64/lib --output-lib test.lib --output-def test.def -o IAWS.dll tpsetunsol_test.o -lwtuxws32_new
C:\MinGW\bin\dllwrap.exe: no export definition file provided.
Creating one, but that may not be what you want
tpsetunsol_test.o: In function `Inittpsetunsol':
d:\dev\IA/tpsetunsol_test.c:13: undefined reference to `tpsetunsol'
collect2.exe: error: ld returned 1 exit status
C:\MinGW\bin\dllwrap.exe: C:\MinGW\bin\gcc exited with status 1


$ nm tpsetunsol_test.o
00000000 b .bss
00000000 d .data
00000000 N .debug_abbrev
00000000 N .debug_aranges
00000000 N .debug_info
00000000 N .debug_line
00000000 N .debug_loc
00000000 r .eh_frame
00000000 t .text
00000004 T _Inittpsetunsol
00000000 T _msghandler@12
         U _tpsetunsol

$ nm ../tuxedo.64/lib/libwtuxws32.a  | grep -i tpsetuns
00000000 I __imp__tpsetunsol@4
00000000 T _tpsetunsol@4

output from gcc preprocessor (-E) (only the line tpsetunsol is declared)

extern void (__attribute__((__stdcall__)) * __attribute__((__stdcall__)) tpsetunsol (void (__attribute__((__stdcall__)) *)(char *, long, long))) (char *, long, long);

I'm guessing but could it be that function pointer declaration in atmi.h is not compatible with GCC ? — knocker_d, Apr 04 '13 at 12:54
There is not enough information in your post. Please, expand it. 1) What's the name of the library (as a file), i.e. `lib*.a` or is it dynamic `*.dll`? 2) Do you link your artifact against it? — Alexander Shukaev, Apr 08 '13 at 12:08
@Haroogan import library names are: wtuxws32.lib and wtuxws32.dll. Yes, I try to link my artifact against that library. — knocker_d, Apr 08 '13 at 12:11
What compiler created these 2 libraries? I guess it's MSVC, isn't it? — Alexander Shukaev, Apr 08 '13 at 12:13
Why both `#define _TMDLLENTRY __stdcall` and `#define _TM_FAR` go after function declaration? MinGW obviously does not mangle name with `@4` because it does not recognize that it should be `__stdcall` since `_TMDLLENTRY` is probably empty when function is declared. — Alexander Shukaev, Apr 08 '13 at 13:29
@Haroogan Sorry for the confusion.. Actually _TMDLLENTRY and _TM_FAR are already defined in another header I just showed them here. I added the the output from the preprocessor (-E switch). Where is the __attribute__ directive coming from ? Also can I just un-mangle the function in the .def file ? — knocker_d, Apr 09 '13 at 06:54
[Attribute](http://gcc.gnu.org/onlinedocs/gcc/Function-Attributes.html) is just the way GCC treats and implements such qualifiers. Extract: _You may also specify attributes with `__` preceding and following each keyword. This allows you to use them in header files without being concerned about a possible macro of the same name. For example, you may use `__noreturn__` instead of `noreturn`._ Of course you cannot simply demangle function neither in `*.def` nor in `lib*.a` files because in the `*.dll` it is still mangled and you will crash in the runtime. MinGW **must** know that it is `stdcall`. — Alexander Shukaev, Apr 09 '13 at 13:22

Alexander Shukaev · Answer 1 · 2013-11-21T03:58:52.567

3

As you have already discovered yourself - name mangling may be different across compilers. To solve your problem follow the instructions:

Download and install (can build from source) gendef utility:
- If you have usual MinGW (targeting 32-bit), then obtain it here;
- If you have MinGW-w64 (targeting 64-bit), then obtain it here.
Run gendef wtuxws32.dll (will generate wtuxws32.def);
Run dlltool -D wtuxws32.dll -d wtuxws32.def -l libwtuxws32.a (will generate libwtuxws32.a);
Put libwtuxws32.a to D:/dev/tuxedo/lib;
Now link against it.

edited Nov 21 '13 at 03:58

answered Apr 08 '13 at 12:36

Alexander Shukaev

16,674
8
70
85

Thanks for the answer. Tried this but it still does not work. gendef generated wtuxws32.def file and resolved tpsetunsol to: tpsetunsol@4 - and this is the same that is found in original wtuxws32.lib. GCC compiler produces tpsetunsol symbol (note the missing @4 in the end). Am I on the right track ? – knocker_d Apr 08 '13 at 13:02
Try to remove both `--add-stdcall-alias` and `--enable-stdcall-fixup`. – Alexander Shukaev Apr 08 '13 at 13:11
no it won't link. Added some output from linker and nm from the library and object file – knocker_d Apr 08 '13 at 13:22

Michael Burr · Accepted Answer · 2013-04-09T15:08:21.210

1

Change the declaration of tpsetunsol() in the atmi.h header from:

extern void (_TMDLLENTRY * _TMDLLENTRY tpsetunsol _((void (_TMDLLENTRY *)(char _TM_FAR *, long, long)))) _((char _TM_FAR *, long, long));

to:

extern  void (_TMDLLENTRY * (_TMDLLENTRY tpsetunsol) _((void (_TMDLLENTRY *)(char _TM_FAR *, long, long)))) _((char _TM_FAR *, long, long));
//                          ^                      ^

so the function will be properly declared to be a stdcall function as it is in the library (as indicated by the @4 suffix on the name in the library).

edited Apr 09 '13 at 15:08

answered Apr 09 '13 at 07:13

Michael Burr

333,147
50
533
760

Thanks. Your suggestion works and I'll accept this although I see that Haroogan was also a big contributor. This declaration was the actual root cause. How come MSVC produces "correct" symbol ? – knocker_d Apr 09 '13 at 12:30
@Michael Burr: Nice one, learned something new today. Is this another inconsistency from MSVC or the implementation whim of GCC? `:)` – Alexander Shukaev Apr 09 '13 at 13:29
@knocker_d, @Haroogan: I've updated the answer with a different modification to the declaration (parens around the function name with the `stdcall` decoration that's associated with it). My original suggestion worked for GCC but not for MSVC. The update works for both compilers. Apparently GCC and MSVC have a slightly different way of binding an attribute/declspec for a function returning a function pointer. I think that in the original declaration from the question, GCC binds the second `_TMDLLENTRY` to the pointer returned from `tpselunsol` (redundantly with the first `_TMDLLENTRY`). – Michael Burr Apr 09 '13 at 15:14

linking dilemma (undefined reference) between MinGW and MSVC. MinGW fails MSVC works

2 Answers2

Linked