fatal error: Python.h: No such file or directory when compiling pybind11 example

Question

I am starting out with pybind11, trying to compile the first example. I am using Xubuntu 20.04. My system python is 3.8, but I have install pybind11 only for python 3.10, which is the version that executes when I type python at the command prompt. When I run the compilation command given in the pybind11 docs:

c++ -O3 -Wall -shared -std=c++11 -fPIC $(python3.10 -m pybind11 --includes) example.cpp -o example$(python3.10-config --extension-suffix)

I get the error message:

     fatal error: Python.h: No such file or directory
  213 | #include <Python.h>

I followed the advice give in the accepted answer to fatal error: Python.h: No such file or directory, and ran

sudo apt install python3.10-dev

but this had no effect, even though Python.h now exists in /usr/include/python3.10. I should perhaps mention that I am not using a virtual environment at this point.

EDIT

python3.10-config --cflags
-I/usr/include/python3.10 -I/usr/include/python3.10  -Wno-unused-result -Wsign-compare -g   -fstack-protector-strong -Wformat -Werror=format-security  -DNDEBUG -g -fwrapv -O2 -Wall

EDIT

I changed the command as suggested by 9769953:

Pybind11Test$ c++ -O3 -Wall -shared -std=c++11 -fPIC $(python3.10 -m pybind11 python3.10-config --includes) example.cpp -o example$(python3.10-config --extension-suffix)
usage: __main__.py [-h] [--includes] [--cmakedir]
__main__.py: error: unrecognized arguments: python3.10-config
example.cpp:1:10: fatal error: pybind11/pybind11.h: No such file or directory
    1 | #include <pybind11/pybind11.h>
      |          ^~~~~~~~~~~~~~~~~~~~~
compilation terminated.

Adding python3.10-config gave me an unrecognized argumnents error, but the compilation failed at a different place. #include <pybind11/pybind11.h> is the first line of example.cpp.

The file exists at

/home/saul/.local/lib/python3.10/site-packages/pybind11/include/pybind11/pybind11.h

EDIT

Latest attempt

Pybind11Test$ c++ -O3 -Wall -shared -std=c++11 -fPIC $(python3.10 -m pybind11 --includes) example.cpp -o example.out
In file included from /home/saul/.local/lib/python3.10/site-packages/pybind11/include/pybind11/detail/../cast.h:13,
                 from /home/saul/.local/lib/python3.10/site-packages/pybind11/include/pybind11/detail/../attr.h:13,
                 from /home/saul/.local/lib/python3.10/site-packages/pybind11/include/pybind11/detail/class.h:12,
                 from /home/saul/.local/lib/python3.10/site-packages/pybind11/include/pybind11/pybind11.h:13,
                 from example.cpp:1:
/home/saul/.local/lib/python3.10/site-packages/pybind11/include/pybind11/detail/../detail/common.h:213:10: fatal error: Python.h: No such file or directory
  213 | #include <Python.h>
      |          ^~~~~~~~~~
compilation terminated.

score 3 · Accepted Answer · answered Jun 05 '22 at 07:28

The problem is that the OS installs the header files for Python 3.10 in /usr/include/python3.10/include. This keeps them separate from the default sytem Python (3.8), in case that development package is also installed (otherwise, one set of files would overwrite the other set of files).

But /usr/include/python3.10/include is not in the default search path for includes for the C++ compiler; you have to add it. You could do that manually, but the python-config tool takes (better) care of that normally. Add

python3.10-config --includes

as a subshell command ($(...)) to the command line. Similar to what pybind does, with $(python -m pybind --includes) to let the compiler find pybind's include files.
Obviously, the correct python-config variant needs to be used, hence python3.10-config.

The shared library files are fewer, and named by having the Python version in their filename, so these can all live safely together in /usr/lib/x86_64-linux-gnu/. You'll find libpython3.8.so and libpython3.10.so next to each other in that directory. So there is no need to add the library search path (that option doesn't even exist for python-config, but it is included in python3.10-config --ldflags).

All in all, the command would then look like:

c++ -O3 -Wall -shared -std=c++11 -fPIC \
$(python3.10-config --includes) \
$(python3.10 -m pybind11 --includes) \
-o example$(python3.10-config --extension-suffix) \
example.cpp

(broken across a few lines for readability.)

fatal error: Python.h: No such file or directory when compiling pybind11 example

1 Answers1