5

I'm developing a Python binding for a C++ library using Boost Python, for Linux and Windows (Visual Studio).

In Windows, the static Boost Python library has a dependency against Python (this is motive for another thread, here), so, in my CMake config I need to do:

if((${CMAKE_SYSTEM_NAME} STREQUAL "Linux") OR APPLE)
     target_link_libraries(my_python_module ${Boost_LIBRARIES})
elseif(WIN32 AND MSVC)
    add_definitions(/DBOOST_PYTHON_STATIC_LIB)
    target_link_libraries(my_python_module ${Boost_LIBRARIES}) #This includes the Boost Python library
    # Even though Boost Python library is included statically, in Windows it has a dependency to the Python library.
    target_link_libraries(my_python_module ${Python_LIBRARIES})
endif()

This works fine in Linux, but in Windows, it only works in Release mode, not in Debug, in which case I always get a:

LINK : fatal error LNK1104: Can't open file 'python37.lib'

After some hair pulling I noticed the issue was caused by CMake instructing Visual Studio to link against 'python37_d.lib' instead of 'python37.lib' in the Debug mode.

However, as I described in the linked issue, the officially provided Boost Python debug library is linked against the Python release library, not the debug one. So, the solution would be to force the link against the Python release library, regardless of the build type. Unfortunately, ${Python_LIBRARIES} sets the library automatically depending on the mode, and I wouldn't like to explicitly specify python37.lib in my code (since I can upgrade Python and I don't want to have to change my CMake scripts because of that).

I found some similar issues here and here, but that doesn't reflect the exact situation I'm facing. Based on those, I tried setting:

target_link_libraries(my_python_module optimized ${Python_LIBRARIES})

But that didn't work either. So, the question is:

Is there a way to force the usage of the Python release library in Debug mode WITHOUT having to set it explicitly and leaving the Python CMake package to do it automatically instead. By explicit I mean doing:

target_link_libraries(my_python_module python37)

Thanks a lot for your help.

Alvaro Palma Aste
  • 381
  • 3
  • 16
  • I don't think there's a way to do this. You could build a debug version of Python to use with a debug version of the module. The problem is that the interpreter and any code linked into it all need to be using with the same C/C++ runtime library (there are both debug and release of of it). – martineau Jun 27 '19 at 22:49
  • 2
    According to `FindBoost.cmake` [here](https://github.com/Kitware/CMake/blob/master/Modules/FindBoost.cmake#L141), you can set a variable `Boost_USE_DEBUG_PYTHON` before calling `find_package(Boost ...)`. Try setting that variable to `OFF` `set(Boost_USE_DEBUG_PYTHON OFF)`. Does that help? – serkan.tuerker Jun 28 '19 at 01:52
  • @martineau, I think you misunderstood my question. I already have the Boost precompiled libraries, and the Boost.Python debug one was indeed compiled linking against the Python release library (`python37.lib`), not the debug one (`python37_d.lib`). But, since `FindPython.cmake` automatically sets the proper Python library for each build type, my code is trying to link against `python37_d.lib` in debug mode, when I have to do it against `python37.lib`. In fact, forcing the linkage against `python37.lib` in debug fixes the issue, but that implies changing my CMake config and I'd prefer not. – Alvaro Palma Aste Jun 28 '19 at 15:25
  • @kanstar, unfortunately not, `Boost_USE_DEBUG_PYTHON` didn't fix the issue. – Alvaro Palma Aste Jun 28 '19 at 15:48
  • 1
    @A.Palma Sorry, I misunderstood your question. With my suggestion, you will only get the release version of `libboost_python` and not `libpython`. Did you have a look at `FindPython.cmake`? The variable `Python_FIND_ABI` looks interesting (see [here](https://github.com/Kitware/CMake/blob/master/Modules/FindPython.cmake#L140)). Try `set(Python_FIND_ABI "OFF" "ANY" "ANY")` (will set debug to `OFF` and malloc and unicode to `ANY`) before doing `find_package(Python)`. – serkan.tuerker Jun 28 '19 at 22:31
  • Of course, that would be too easy: Python ships with pyconfig set to use pragmas to inject a pythonx_d.lib dependency: https://github.com/python/cpython/blob/3.10/PC/pyconfig.h#L270-L276 – kfsone Nov 09 '21 at 22:15

1 Answers1

5

It seems that set(Python_FIND_ABI "OFF" "ANY" "ANY") as suggested in the comments by kanstar would be the correct way to do this. However, while Python_FIND_ABI is in CMake master, it hasn't been released yet in the latest version (v3.15.2 as of this writing).

In the meantime, there are solutions dependent on the CMake version.

CMake 3.12 and above

It's possible to link against FindPython's Python_LIBRARY_RELEASE, which isn't meant to be part of the module's public interface, but the variable is set correctly nonetheless.

cmake_minimum_required (VERSION 3.12)
find_package(Python ..<choose your COMPONENTS; refer to FindPython docs>..)
if(WIN32 AND MSVC)
  target_link_libraries(my_python_module ${Python_LIBRARY_RELEASE})
endif()

CMake 3.0.4 to 3.11

Thanks to a comment by @Phil, we can expand the answer to include earlier CMake versions which had the FindPythonLibs module that sets the PYTHON_LIBRARY_RELEASE variable instead.

cmake_minimum_required (VERSION 3.0)
find_package(PythonLibs ..<refer to FindPythonLibs docs>..)
if(WIN32 AND MSVC)
  target_link_libraries(my_python_module ${PYTHON_LIBRARY_RELEASE})
endif()
  • 1
    I believe that should be ``${PYTHON_LIBRARY_RELEASE}`` in all caps. – Phil Sep 27 '19 at 19:55
  • @Phil Is there a case where `Python_LIBRARY_RELEASE` doesn't work? `Python_*`, `Python2_*`, and `Python3_*` are the prefixes to use according to [FindPython docs](https://cmake.org/cmake/help/latest/module/FindPython.html). –  Sep 30 '19 at 14:42
  • @Phil Ah, I think you might be referring to [FindPythonLibs](https://cmake.org/cmake/help/v3.15/module/FindPythonLibs.html), which does have the token `PYTHON_LIBRARY_RELEASE` ([source](https://gitlab.kitware.com/cmake/cmake/blob/release/Modules/FindPythonLibs.cmake)), but the entire module's deprecated since 3.12. –  Oct 01 '19 at 14:37
  • 1
    Yes, I think you are right. This is what the pybind11 example uses, and I am blindly following that. Thanks for your help and the excellent answer! – Phil Oct 01 '19 at 20:56
  • what about visual studio? – Nickpick Nov 30 '19 at 12:57
  • @Nickpick assuming for example Python 3.7; in Visual Studio go to -> Properties -> (top left) set Configuration: Debug. Then in Linker -> Input, make sure the Additional Dependencies field has `python37.lib` and not `python37_d.lib` –  Dec 14 '19 at 14:38
  • @Konafa, yes, but that won't be reflected in CMakeList, but would only work if used with visual studio solution file – Nickpick Dec 14 '19 at 14:54
  • The original answer already covers cmake. I probably misunderstood your question then; help me understand what you mean. –  Dec 14 '19 at 16:36
  • 1
    Windows 10, VS2019, Python 3.10: this doesn't work, with Py310 installed into C:\Python310: ``` LIST(INSERT CMAKE_PREFIX_PATH 0 "C:\\Python310") SET(Python_USE_SHARED_LIBS OFF) SET(Python_IMPLEMENTATIONS CPython) SET(Python_FIND_ABI "OFF" "ANY" "ANY") FIND_PACKAGE(Python3 3.10.0 REQUIRED COMPONENTS Development.embed) ``` fails to find python in debug build, ``` – kfsone Nov 09 '21 at 22:38