6

I'm having trouble understanding how to use pybind11 conan package. I can use some others, but pybind11 is giving me hard time.

My starting point is as follows:

conanfile.txt:

[requires]
pybind11/2.7.1

[generators]
cmake

main.cpp:

#include <pybind11/pybind11.h>

int add(int i, int j) {return i + j;}

PYBIND11_MODULE(cobind, m) {m.def("add", &add);}

CMakeLists.txt

cmake_minimum_required(VERSION 3.21)
project(cobind11 VERSION 1.0 LANGUAGES CXX)

include(${CMAKE_BINARY_DIR}/conanbuildinfo.cmake)
conan_basic_setup()

pybind11_add_module(cobind main.cpp)

The log of my suffering trying to make this work:

Starting point

If I try to build as above the project I get the following error:

CMake Error at CMakeLists.txt:10 (pybind11_add_module):
  Unknown CMake command "pybind11_add_module".

-- Configuring incomplete, errors occurred!

Adding find_package(pybind11 REQUIRED)

If I add a line find_package(pybind11 REQUIRED) I get the following error:

CMake Error at CMakeLists.txt:16 (find_package):
  By not providing "Findpybind11.cmake" in CMAKE_MODULE_PATH this project has
  asked CMake to find a package configuration file provided by "pybind11",
  but CMake did not find one.

Adding include([...]/pybind11Tools.cmake)

If I add a line #include(${CONAN_PYBIND11_ROOT}/lib/cmake/pybind11/pybind11Tools.cmake) I get the following error:

CMake Error at /home/[...]/pybind11Tools.cmake:100 (set_property):
  set_property could not find TARGET pybind11::pybind11.  Perhaps it has not yet been created.

like in this issue https://github.com/pybind/pybind11/issues/3388. Maybe it is fixed in the new release?

Upgrading to 2.8.1

The fresh pybind11 is 2.8.1, let's upgrade

pybind11/2.8.1: Not found in local cache, looking in remotes...
pybind11/2.8.1: Trying with 'conancenter'...
ERROR: Unable to find 'pybind11/2.8.1' in remotes

OK. One can read between the lines, that it used to work, so maybe let's downgrade?

Downgrading to 2.4.3

If I require pybind/2.4.3 instead of pybind/2.7.1 in conanfile.txt I get

fatal error: Python.h: No such file or directory
  112 | #include <Python.h>
      |          ^~~~~~~~~

like in this issue https://github.com/pybind/pybind11/issues/1781. Differently from that issue, installing python*-dev does not help. But whatever, I don't want to use old pybind anyway.

Trying a conanfile from test package

The conancentral recipes contain a test package (https://github.com/conan-io/conan-center-index/tree/master/recipes/pybind11/all/test_package), which is tested automatically when the package is created. Let's try that instead!

CMake Error at CMakeLists.txt:13 (find_package):
  By not providing "Findpybind11.cmake" in CMAKE_MODULE_PATH this project has
  asked CMake to find a package configuration file provided by "pybind11",
  but CMake did not find one.

Am I just hopeless???

Maybe, but! I can:

Build the https://github.com/pybind/cmake_example
Build the https://github.com/pybind/python_example
Build the conancentral pybind11 recipe! The same that fails when I extract the test package to a separate folder. Whaaat??

conan is torturing me

FreshD
  • 2,914
  • 2
  • 23
  • 34
psarka
  • 1,562
  • 1
  • 13
  • 25
  • Try adding `find_package(pybind11 REQUIRED)` before the line containing `pybind11_add_module(cobind main.cpp)`. – darcamo Nov 03 '21 at 17:39
  • Unfortunately it does not work with a different error (I'll paste in the post). However, in the meantime I found this PR in pybind11 https://github.com/pybind/pybind11/pull/3420, so it might be that things will start working in the next release. – psarka Nov 04 '21 at 04:25
  • In the "conanbuildinfo.cmake" file generated by conan, that you are including in your CMakeLists.txt file, many variables are set for the different dependencies. Particularly, you get a "CONAN_LIB_DIRS_" variable for each dependency with the proper library folder of that dependency. That means you can use `${CONAN_LIB_DIRS_PYBIND11}/cmake/pybind11/pybind11Tools.cmake` to get the full path of that file. – darcamo Nov 05 '21 at 13:06
  • Thanks @darcamo, but I can't make this work either. I updated my post, this whole thing is super infuriating. – psarka Nov 06 '21 at 22:09
  • 1
    The pybind issue you posted is for multi-config generators, but you are using the cmake generator. Therefore, it is not related to your problems. – darcamo Nov 08 '21 at 13:29
  • 1
    The most recent version of pybind available in conan is 2.7.1 (do a `conan search pybind11 -r=all` to check). You can create an issue (or a pull-request) [here](https://github.com/conan-io/conan-center-index) to add a new version. – darcamo Nov 08 '21 at 13:31

1 Answers1

2

I have used pybind in the past (two or three years ago) and it worked without problems with conan. However, I tried to install it now and faced similar problems. This might be related to the evolution of conan towards conan 2.0 (an alpha version was released today) and that the recipe was not updated to account changes.

An alternative that you might consider, is to install pybind with conan using a different generator. More specifically, the CMakeDeps generator. With the CMakeDeps generator conan will create a pybind11-config.cmake file for you and you just need to use find_package(pybind11 REQUIRED) in CMakeLists.txt. That is, there is no "conan specific stuff" in your CMakeLists.txt file.

conanfile.txt

[requires]
pybind11/2.7.1

[generators]
CMakeDeps

CMakeLists.txt

cmake_minimum_required(VERSION 3.21)
project(cobind11 VERSION 1.0 LANGUAGES CXX)

# Tell cmake to also search in the buld folder for the "<library>-config.cmake"
# files
list(APPEND CMAKE_PREFIX_PATH "${CMAKE_BINARY_DIR}")
find_package(pybind11 REQUIRED)

pybind11_add_module(cobind main.cpp)

main.cpp

#include <pybind11/pybind11.h>

int add(int i, int j) { return i + j; }

PYBIND11_MODULE(cobind, m) { m.def("add", &add); }

With this I was able to build the library and use it in Python.

Conan deeper integration with CMake

You can also use the CMakeToolchain generator in addition to the CMakeDeps generator. It generates a conan_toolchain.cmake file that you pass to the cmake command with --toolchain conan_toolchain.cmake. If you use it, there is no need to add the list(APPEND CMAKE_PREFIX_PATH "${CMAKE_BINARY_DIR}") line to your CMakeLists.txt file. Furthermore, the settings you specify in conan, such as build type and compiler, will impact cmake. That is, you don't need to specify these things in both conan and cmake. This seems to be where conan is going, regarding cmake integration, in the comming 2.0 release.

darcamo
  • 3,294
  • 1
  • 16
  • 27