1

I am working on a Windows 10 machine 64-bit installation with python 3.7.9 installed (not under Anaconda) in the typical "Program Files" directory and pybind11 installed in my working directory. The goal here is simply to get pybind11 running from C++. The program compiles fine, but the executable doesn't run, giving the error "The code execution cannot proceed because python37.dll was not found." Reinstalling the program may fix this problem." However, the python37.dll (and the debug version) file locations are in the CMakeCache.txt file generated (included below along with other details). I tried putting quotes around the file paths in CMakeCache.txt in hopes that the error was simply due to the space in the "Program Files" directory name, and have the short code versions of "Program Files" in the environment path variables (PROGRA~1), but the error persists. I am new to Cmake and C++. What am I missing?

The other related posts I found addressed moving files (https://www.reddit.com/r/techsupport/comments/c2z5rl/python37dll_not_found_but_i_find_it_in_windows/, which is not the issue here), or a system where python was not installed (python37.dll not linked in executable, again not the case here), problems with the Anaconda installation (Why do I get a python37.dll error when starting jupyter notebook via conda, but I'm not running Anaconda), or other software I'm not using. All the others links point to download sites.

There were no errors in the installation processes of either program. The program directories, relevant codes, build commands and output are given below.

From the ./NewKamodo/build directory, by commands at the cmd prompt with the output are (I have replaced my internal structure with ... for privacy):

...\NewKamodo\build>cmake .. -G"Visual Studio 15 2017" -A x64  
-- Selecting Windows SDK version 10.0.17763.0 to target Windows 10.0.19041.  
-- The C compiler identification is MSVC 19.16.27045.0  
-- The CXX compiler identification is MSVC 19.16.27045.0  
-- Detecting C compiler ABI info  
-- Detecting C compiler ABI info - done  
-- Check for working C compiler: C:/Program Files (x86)/Microsoft Visual Studio/2017/BuildTools/VC/Tools/MSVC/14.16.27023/bin/Hostx86/x64/cl.exe - skipped  
-- Detecting C compile features  
-- Detecting C compile features - done  
-- Detecting CXX compiler ABI info  
-- Detecting CXX compiler ABI info - done  
-- Check for working CXX compiler: C:/Program Files (x86)/Microsoft Visual Studio/2017/BuildTools/VC/Tools/MSVC/14.16.27023/bin/Hostx86/x64/cl.exe - skipped  
-- Detecting CXX compile features  
-- Detecting CXX compile features - done  
-- pybind11 v2.6.3 dev1  
-- Found PythonInterp: C:/Program Files/Python37/python.exe (found version "3.7.9")  
-- Found PythonLibs: C:/Program Files/Python37/libs/python37.lib  
-- Performing Test HAS_MSVC_GL_LTCG  
-- Performing Test HAS_MSVC_GL_LTCG - Success  
-- Found Python3: C:/Program Files/Python37/python.exe (found version "3.7.9") found components: Interpreter Development Development.Module Development.Embed  
-- Configuring done  
-- Generating done  
-- Build files have been written to: .../NewKamodo/build  

...\NewKamodo\build>cmake --build .  
Microsoft (R) Build Engine version 15.9.21+g9802d43bc3 for .NET Framework  
Copyright (C) Microsoft Corporation. All rights reserved.  

  Checking Build System  
  Building Custom Rule .../NewKamodo/CMakeLists.txt  
  KamodoCXX.cpp  
  KamodoCXX.vcxproj -> ...\NewKamodo\build\Debug\KamodoCXX.exe  
  Building Custom Rule .../NewKamodo/CMakeLists.txt  

...\NewKamodo\build>cd Debug  
...\NewKamodo\build\Debug>KamodoCXX.exe

pop up window error: "The code execution cannot proceed because python37.dll was not found. Reinstalling the program may fix this problem."

My program directory structure is:
NewKamodo

  • ./build
  • ./pybind11
  • CMakeLists.txt
  • KamodoCXX.cpp

The C++ code in KamodoCXX.cpp is the hello world example taken from https://pybind11.readthedocs.io/en/stable/advanced/embedding.html and copied below.

#include <pybind11/embed.h> // everything needed for embedding
namespace py = pybind11;

int main() {
    py::scoped_interpreter guard{}; // start the interpreter and keep it alive

    py::print("Hello, World!"); // use the Python API
}

The CMakeLists.txt file is also simple:

cmake_minimum_required(VERSION 3.4)
project(KamodoCXX)

#specify the C++ standard
set(CMAKE_CXX_STANDARD 11)
set(CMAKE_CXX_REQUIRED True)

add_subdirectory(pybind11)
find_package(Python3 COMPONENTS Interpreter Development)

add_executable(KamodoCXX KamodoCXX.cpp)
target_link_libraries(KamodoCXX PRIVATE pybind11::embed)

CMakeCache.txt contents (python portion of file only for brevity):
...

PYBIND11_INCLUDE_DIR:INTERNAL=.../NewKamodo/pybind11/include
//ADVANCED property for variable: PYTHON_EXECUTABLE
PYTHON_EXECUTABLE-ADVANCED:INTERNAL=1
PYTHON_INCLUDE_DIRS:INTERNAL=C:/Program Files/Python37/include
PYTHON_IS_DEBUG:INTERNAL=0
PYTHON_LIBRARIES:INTERNAL=C:/Program Files/Python37/libs/python37.lib
PYTHON_MODULE_EXTENSION:INTERNAL=.cp37-win_amd64.pyd
PYTHON_MODULE_PREFIX:INTERNAL=
PYTHON_VERSION:INTERNAL=3.7.9
PYTHON_VERSION_MAJOR:INTERNAL=3  
PYTHON_VERSION_MINOR:INTERNAL=7  
Python_ADDITIONAL_VERSIONS:INTERNAL=3.10;3.9;3.8;3.7;3.6;3.5;3.4  
//CMAKE_INSTALL_PREFIX during last run  
_GNUInstallDirs_LAST_CMAKE_INSTALL_PREFIX:INTERNAL=C:/Program Files/KamodoCXX  
_Python:INTERNAL=PYTHON  
_Python3_DEVELOPMENT_EMBED_SIGNATURE:INTERNAL=78025eb1b345e021c9dadb673f40d7b6  
_Python3_DEVELOPMENT_MODULE_SIGNATURE:INTERNAL=78025eb1b345e021c9dadb673f40d7b6  
//Path to a program.  
_Python3_EXECUTABLE:INTERNAL=C:/Program Files/Python37/python.exe  
//Path to a file.  
_Python3_INCLUDE_DIR:INTERNAL=C:/Program Files/Python37/include  
//Python3 Properties  
_Python3_INTERPRETER_PROPERTIES:INTERNAL=Python;3;7;9;64;;cp37-win_amd64;C:\Program Files\Python37\Lib;C:\Program Files\Python37\Lib;C:\Program Files\Python37\Lib\site-packages;C:\Program Files\Python37  \Lib\site-packages  
_Python3_INTERPRETER_SIGNATURE:INTERNAL=d8451d76c667fb34c5414ba253c4dadd  
//Path to a library.  
_Python3_LIBRARY_DEBUG:INTERNAL=C:/Program Files/Python37/libs/python37_d.lib  
//Path to a library.  
_Python3_LIBRARY_RELEASE:INTERNAL=C:/Program Files/Python37/libs/python37.lib  
//Path to a library.  
_Python3_RUNTIME_LIBRARY_DEBUG:INTERNAL=C:/Program Files/Python37/python37_d.dll  
//Path to a library.  
_Python3_RUNTIME_LIBRARY_RELEASE:INTERNAL=C:/Program Files/Python37/python37.dll  
//True if pybind11 and all required components found on the system
pybind11_FOUND:INTERNAL=TRUE  
//Directory where pybind11 headers are located  
pybind11_INCLUDE_DIR:INTERNAL=.../NewKamodo/pybind11/include  
//Directories where pybind11 and possibly Python headers are located  
pybind11_INCLUDE_DIRS:INTERNAL=.../NewKamodo/pybind11/include;C:/Program Files/Python37/include
Rebecca Ringuette
  • 11
  • 4
  • Does the bitness (32/64) of the Python DLL match the compile target? Windows will often say *DLL not found* when you can see perfectly well it is there if there is a mismatch. – BoarGules Feb 05 '21 at 08:14
  • According to https://stackoverflow.com/questions/3411079/why-does-the-python-2-7-amd-64-installer-seem-to-run-python-in-32-bit-mode, the sys.maxsize indicates that my python installation is 64-bit (3.7.9). My machine is also 64-bit according to the lines printed when running python from the command line. Also, I believe the x64 at the end of my first cmake command forces the target to be compiled as a 64-bit target (right?). Is this method correct? – Rebecca Ringuette Feb 06 '21 at 00:31
  • For Python you are correct, though printing `sys.version` is more direct. For `cmake` it looks right but I'm no expert there. So, one possible cause eliminated. Best of luck. – BoarGules Feb 06 '21 at 07:17
  • @RebeccaRinguette I think you should read this first https://stackoverflow.com/questions/2463243/dll-search-on-windows . So you should either copy python37.dll to the folder where is your executable located or update the PATH environment variable to point to the folder where python37.dll is present. Latter is better, IMO. – Sergei Nikulov Feb 06 '21 at 12:42
  • @SergeiNikulov The PATH environment variable already points to the directory where python37.dll is present. I'm starting to wonder if the problem is that Window 10 has a lot of redirects in place. Using Dependency Walker (https://stackoverflow.com/questions/1993673/what-is-the-equivalent-of-linuxs-ldd-on-windows) points out the breaks, but also doesn't compensate for the redirects. Any thoughts on this? – Rebecca Ringuette Feb 08 '21 at 16:57
  • Then www.dependencywalker.com tool is your best friend. – Sergei Nikulov Feb 08 '21 at 17:01
  • (I got the Windows 10 redirect from this: https://social.msdn.microsoft.com/Forums/en-US/a28331ae-19a3-4a34-b3ba-1e8fd4430375/missing-apimswincore-dlls?forum=windowsgeneraldevelopmentissues) – Rebecca Ringuette Feb 08 '21 at 18:06

3 Answers3

0

Here's what we used to do with python 2.7 but it might work for 3+. We obtain the full path to the python libraries by running python from cmake.

add_subdirectory(pybind11)
find_package(Python3 COMPONENTS Interpreter Development)
FIND_PACKAGE(PythonLibs "${PYTHON_VERSION_MAJOR}.${PYTHON_VERSION_MINOR}" REQUIRED)
execute_process(COMMAND ${PYTHON_EXECUTABLE} -c "import sys; from distutils.sysconfig import get_python_lib; print(get_python_lib().split(sys.prefix)[-1][1:])"
                    OUTPUT_VARIABLE PYTHON_SITE_PACKAGES OUTPUT_STRIP_TRAILING_WHITESPACE)

*edited for python3

Asher Pembroke
  • 71
  • 1
  • 3
  • I already had the first two lines in my CMakeLists.txt file. After adding the last two and recompiling, I get different feedback while compiling (shown below), but the same error when trying to use the executable. It looks like the python command worked because I see no trailing white space, although there wasn't any in the CMakeCache.txt file before. -- Found PythonLibs: optimized;C:/Program Files/Python37/libs/python37.lib;debug;C:/Program Files/Python37/libs/python37_d.lib (found suitable version "3.7.9", minimum required is "3.7") – Rebecca Ringuette Feb 04 '21 at 22:09
0

I worked with the Pybind11 Developers on this BUG post to achieve a final solution, which I summarize below. The alternative error that popped up along the way was:

Fatal Python Error: initfsencoding: unable to load the file system codec
ModuleNotFoundError: No module named ‘encodings’

when attempting to run the executable from the command prompt.

https://github.com/pybind/pybind11/issues/2855#issuecomment-777648383

On Windows 10 (without using windows subsystem for linux):

  • Install either Anaconda or miniconda
  • build a conda environment with the commands
conda create -n embedtest -c conda-forge python==3.7.9 pybind11==2.6.2 cmake=3.19.4
conda activate embedtest

Note: Any version of cmake later than 3.15 seems to work. I didn't specify which version of either pybind11 or cmake, and used cmake 3.19.4 successfully. (Pybind11 version 2.6.2 is the current default). Also, avoid python versions of 3.8 or 3.9 as those are known not to work (see pybind11 docs).

CMakeLists.txt contents:

cmake_minimum_required(VERSION 3.15)
project(main)

find_package(Python COMPONENTS Interpreter Development) 
find_package(pybind11 REQUIRED)

add_executable(main main.cpp)
target_link_libraries(main pybind11::embed)
  • Next run the commands below from a windows command prompt where the conda environment is activated and you have moved to the same directory as the main.cpp and CMakeLists.txt files are located. Note that the -B command builds the output in the subdirectory called build.
cmake -B build -A x64
cmake --build build
set PYTHONHOME=C:\Users\test\Miniconda3\venvs\embedtest
build\Debug\main.exe

Note: The PYTHONHOME variable MUST be set from the terminal, as setting this variable with cmake does not make the new definition available at build time or execution time, causing errors. The directory example given here is the location of the python executable in the conda environment, which can be easily obtained by adding the following line to the CMakeLists.txt code after the python package has been found using find_package(Python COMPONENTS ....)

message(STATUS ${Python_EXECUTABLE})

If your goal is to use pybind11 without a virtual environment, then the set(Python_VIRTUALENV ONLY) line will not be necessary, but the python version called will be the default one installed.

If you wish to use pybind11 on Windows 10 via WSL, then the instructions are almost identical. The CMakeLists.txt and main.cpp files are the same, but the terminal commands are slightly different:

cmake -B build
cmake --build build
cd build
chmod +x main
./main
Rebecca Ringuette
  • 11
  • 4
0

I got this error using the MSVC compiler and the version of python 3.10 available for download through python.org, and was similarly confused because I was using the 64 bit MSVC compiler with the 64 bit python 3.10 library. If it's a bug in pybind11, it still hasn't been fixed (I'm using the master branch as of today). However, I was able to get around the bug without switching to conda by using the mingw compiler instead, i.e.:

cmake -S . -B builddir -G "MinGW Makefiles"
cmake --build builddir
Catherine Holloway
  • 739
  • 1
  • 7
  • 20