2

I need to build a c extension for Python for Windows using distutils. My setup.py looks like this:

My machine is: Windows Server 2019 64 bit. Python 3.9

(I need to include mysql and a couple of other libraries. I am using the MSVC compiler).

from setuptools import setup, Extension
import shutil
import os, platform


include_dirs=[
r"C:\src\vcpkg\packages\zlib_x86-windows\bin",
r"C:\src\vcpkg\packages\zlib_x86-windows\lib",
r"C:\src\vcpkg\packages\zlib_x86-windows",
r"C:\src\vcpkg\packages\zlib_x86-windows\include",
r"C:\vcpkg\vcpkg\packages\picojson_x86-windows\include\picojson",
r"C:\vcpkg\vcpkg\packages\picojson_x86-windows\include",
r"C:\vcpkg\vcpkg\packages\picojson_x86-windows",
r"C:\libjwt\include",
r"C:\libjwt\build\libjwt\Debug",
r"C:\src\vcpkg\packages\libmysql_x64-windows\include\mysql",
r"C:\src\vcpkg\packages\libmysql_x64-windows\lib",
r"C:\vcpkg\vcpkg\packages\libmysql_x64-windows\bin",
]
library_dirs=[
r"C:\src\vcpkg\packages\libmysql_x64-windows\debug\lib",
r"C:\libjwt\build\libjwt\Debug",
r"C:\vcpkg\vcpkg\packages\libmysql_x64-windows\bin",
]

FLD=Extension('Download', 
libraries=['mysqlclient', 'zlib', 'jwt'], 
library_dirs=include_dirs,
include_dirs=include_dirs,
sources=['Download.c'],language='c')

setup(
      name='Download', version='1.0', ext_modules=[FLD]
    )

platform.platform()

When I run python setup.py install, I get:

running install
running bdist_egg
running egg_info
writing Download.egg-info\PKG-INFO
writing dependency_links to Download.egg-info\dependency_links.txt
writing top-level names to Download.egg-info\top_level.txt
reading manifest file 'Download.egg-info\SOURCES.txt'
writing manifest file 'Download.egg-info\SOURCES.txt'
installing library code to build\bdist.win-amd64\egg
running install_lib
running build_ext
building 'Download' extension

C:\Program Files (x86)\Microsoft Visual Studio\2017\Professional\VC\Tools\MSVC\14.16.27023\bin\HostX86\x64\cl.exe /c /nologo /Ox /W3 /GL /DNDEBUG /MD -IC:\src\vcpkg\packages\zlib_x86-windows\bin -IC:\src\vcpkg\packages\zlib_x86-windows\lib -IC:\src\vcpkg\packages\zlib_x86-windows -IC:\src\vcpkg\packages\zlib_x86-windows\include -IC:\vcpkg\vcpkg\packages\picojson_x86-windows\include\picojson -IC:\vcpkg\vcpkg\packages\picojson_x86-windows\include -IC:\vcpkg\vcpkg\packages\picojson_x86-windows -IC:\libjwt\include -IC:\libjwt\build\libjwt\Debug -IC:\Program Files\MySQL\MySQL Server 5.0\include -IC:\Program Files\Python39\include -IC:\Program Files\Python39\include -IC:\Program Files (x86)\Microsoft Visual Studio\2017\Professional\VC\Tools\MSVC\14.16.27023\ATLMFC\include -IC:\Program Files (x86)\Microsoft Visual Studio\2017\Professional\VC\Tools\MSVC\14.16.27023\include -IC:\Program Files (x86)\Windows Kits\NETFXSDK\4.6.1\include\um -IC:\Program Files (x86)\Windows Kits\10\include\10.0.17763.0\ucrt -IC:\Program Files (x86)\Windows Kits\10\include\10.0.17763.0\shared -IC:\Program Files (x86)\Windows Kits\10\include\10.0.17763.0\um -IC:\Program Files (x86)\Windows Kits\10\include\10.0.17763.0\winrt -IC:\Program Files (x86)\Windows Kits\10\include\10.0.17763.0\cppwinrt /TcFLdownload.c /Fobuild\temp.win-amd64-3.9\Release\Download.obj
Download.c

[A bunch of warnings related to sprintf]

C:\Program Files (x86)\Microsoft Visual Studio\2017\Professional\VC\Tools\MSVC\14.16.27023\bin\HostX86\x64\link.exe /nologo /INCREMENTAL:NO /LTCG /DLL /MANIFEST:EMBED,ID=2 /MANIFESTUAC:NO /LIBPATH:C:\src\vcpkg\packages\zlib_x86-windows\bin /LIBPATH:C:\src\vcpkg\packages\zlib_x86-windows\lib /LIBPATH:C:\src\vcpkg\packages\zlib_x86-windows /LIBPATH:C:\src\vcpkg\packages\zlib_x86-windows\include /LIBPATH:C:\vcpkg\vcpkg\packages\picojson_x86-windows\include\picojson /LIBPATH:C:\vcpkg\vcpkg\packages\picojson_x86-windows\include /LIBPATH:C:\vcpkg\vcpkg\packages\picojson_x86-windows /LIBPATH:C:\libjwt\include /LIBPATH:C:\libjwt\build\libjwt\Debug /LIBPATH:C:\Program Files\MySQL\MySQL Server 5.0\include /LIBPATH:C:\Program Files\Python39\libs /LIBPATH:C:\Program Files\Python39\PCbuild\amd64 /LIBPATH:C:\Program Files (x86)\Microsoft Visual Studio\2017\Professional\VC\Tools\MSVC\14.16.27023\ATLMFC\lib\x64 /LIBPATH:C:\Program Files (x86)\Microsoft Visual Studio\2017\Professional\VC\Tools\MSVC\14.16.27023\lib\x64 /LIBPATH:C:\Program Files (x86)\Windows Kits\NETFXSDK\4.6.1\lib\um\x64 /LIBPATH:C:\Program Files (x86)\Windows Kits\10\lib\10.0.17763.0\ucrt\x64 /LIBPATH:C:\Program Files (x86)\Windows Kits\10\lib\10.0.17763.0\um\x64 mysqlclient.lib zlib.lib jwt.lib /EXPORT:Download build\temp.win-amd64-3.9\Release\Download.obj /OUT:build\lib.win-amd64-3.9\Download.cp39-win_amd64.pyd /IMPLIB:build\temp.win-amd64-3.9\Release\Download.cp39-win_amd64.lib
   Creating library build\temp.win-amd64-3.9\Release\Download.cp39-win_amd64.lib and object build\temp.win-amd64-3.9\Release\Download.cp39-win_amd64.exp
Download.obj : error LNK2001: unresolved external symbol jwt_free
Download.obj : error LNK2001: unresolved external symbol jwt_get_grants_json
Download.obj : error LNK2001: unresolved external symbol jwt_decode
mysqlclient.lib(my_init.obj) : error LNK2001: unresolved external symbol __imp_RegCloseKey
mysqlclient.lib(my_init.obj) : error LNK2001: unresolved external symbol __imp_RegEnumValueA
mysqlclient.lib(my_init.obj) : error LNK2001: unresolved external symbol __imp_RegOpenKeyExA
mysqlclient.lib(common.obj) : error LNK2001: unresolved external symbol __imp_EqualSid
mysqlclient.lib(common.obj) : error LNK2001: unresolved external symbol __imp_GetTokenInformation
mysqlclient.lib(common.obj) : error LNK2001: unresolved external symbol __imp_IsValidSid
mysqlclient.lib(common.obj) : error LNK2001: unresolved external symbol __imp_LookupAccountNameW
mysqlclient.lib(random.obj) : error LNK2001: unresolved external symbol __imp_CryptAcquireContextA
mysqlclient.lib(random.obj) : error LNK2001: unresolved external symbol __imp_CryptReleaseContext
mysqlclient.lib(random.obj) : error LNK2001: unresolved external symbol __imp_CryptGenRandom
build\lib.win-amd64-3.9\Download.cp39-win_amd64.pyd : fatal error LNK1120: 13 unresolved externals
error: command 'C:\\Program Files (x86)\\Microsoft Visual Studio\\2017\\Professional\\VC\\Tools\\MSVC\\14.16.27023\\bin\\HostX86\\x64\\link.exe' failed with exit code 1120

I am using vcpkg to download and install libmysql and zlib. There is no c jwt in vcpkg so I built it manually.

I don't know whether this has to do with 32 vs 64 bits, library versions or something else. I am not a c programmer so I am a bit lost.

I got the mysql lib by: vcpkg install libmysql:64-windows, so I have the 64 bit version of libmysql.lib. And I am pointing to the header of the same version (downloaded from vcpkg). I guess that the python distutils c compiler and Vcpkg are using the same compiler, which is MVS 2017.

I have spent hours on these linking errors, and I can't find the solution. I would appreciate any help.

Jorge
  • 1,350
  • 2
  • 10
  • 19
  • It looks as if you forgot to pass some library names to your linker. e.g. https://stackoverflow.com/q/22954119/5769463 or https://stackoverflow.com/a/30651895/5769463 – ead Nov 15 '21 at 18:29
  • But isn't this handled by the python setup tools from the distutils package? I don't get to compose the link arguments. At least I haven't found a way to do so. – Jorge Nov 16 '21 at 10:16
  • distutils command line depends on the exact toolchain with which the python was built, which is possible not the one you are using. just add `advapi32` to `libraries`-list. – ead Nov 16 '21 at 11:07
  • Thanks! I did and it fixed the libmysql link errors. Now I am still left with the jwt link errors: Download.obj : error LNK2001: unresolved external symbol jwt_free Download.obj : error LNK2001: unresolved external symbol jwt_get_grants_json Download.obj : error LNK2001: unresolved external symbol jwt_decode – Jorge Nov 16 '21 at 12:27
  • There is something possible wrong with your jwt-library: it is called differently (often debug-build adds d-suffix), it is x86 and not x86-64, other issues. Use dumpbin to see the symbols in the library (here is an example how one could detect/trouble-shout x86/x86-64 mismatch) or add verbose flag for linking step to see what is going on/wrong. – ead Nov 16 '21 at 12:59
  • Example: https://stackoverflow.com/a/63757694/5769463 – ead Nov 16 '21 at 13:14
  • Prior to you comment I tried changing jwt for libjwt (the name in github) and it could not find the library, so I think it should be jwt. The .lib and the heather files are jwt.lib and jwt.h and are both present in the specified folders. I also checked that the function names do exist in the header (i.e. jwt_free) and I am pretty sure their signature match my source code. I will dig into the dumpbin tool and link you posted. Thanks – Jorge Nov 16 '21 at 13:36
  • (I also tried the verbose linking but there is no new message for the jwt error) – Jorge Nov 16 '21 at 13:47
  • So I used the dumpbin tool and the symbols I get are prepended with an '_' before. So for example on entry of dumpbin says: 'type () External | _jwt_freemem', where in my c code I use jwt_freemem. Do you think this might be the issue? How could I circumvent it? – Jorge Nov 16 '21 at 13:55
  • As the link I have provided (https://stackoverflow.com/a/63757694/5769463) explains: if symbols are prepended with `_` it is 32bit-library and cannot be used for 64bit builds. See link for more information. – ead Nov 16 '21 at 13:59
  • Sorry I don't have much experience with c. That jwt lib was just compiled by me in the same 64-bit machine so I though I was generating a 64-bit library since I wasn't getting the 32 to 64 bit conflict warning shown in your link. I will try to build it for 64 bit. – Jorge Nov 16 '21 at 14:06
  • Ok, I fixed those by building the x64 bit jwt.lib but that unveild a bunch of new link errors, for example: LINK : warning LNK4098: defaultlib 'MSVCRTD' conflicts with use of other libs; use /NODEFAULTLIB:library jwt.lib(jwt.obj) : error LNK2001: unresolved external symbol json_object jwt.lib(jwt.obj) : error LNK2001: unresolved external symbol json_false jwt.lib(jwt.obj) : error LNK2001: unresolved external symbol json_delete jwt.lib(jwt-openssl.obj) : error LNK2001: unresolved external symbol BIO_set_flags Amongst many other similar ones. – Jorge Nov 16 '21 at 14:41
  • If you answer I will upvote it, by the way – Jorge Nov 16 '21 at 14:42
  • `library_dirs=include_dirs` - this doesn't look right. Do you mean `library_dirs=library_dirs`? – DavidW Nov 17 '21 at 20:12
  • Yeah that was some playing around that I was doing. But all LINK errors still remain. when changing that. Still I don't understand why so many LiNK unresolved externals. When I run dumpbin /SYMBOLS on libmysql I get almost nothing, no functions. But when I run it on jwt.lib I get all the functions, including all that the LINK complains about not existing. – Jorge Nov 18 '21 at 09:37

0 Answers0