How to create a Python package from a PyBind11 module created with CMake

Question

I have a C++ code using Cmake & make for the building part. I also have python bindings to this code thanks to PyBind11.

I use pybind11_add_module inside CMakeLists.txt and now when I build (cmake -Bbuild && cd build && make) it creates a python_module.so in the build directory.

I can move it manually elsewhere, and if the module is in the same folder I can use it in python scripts.

Now I'd like to "package" this module and I don't know how. My goal would be to be able to pip install python_package_from_module to use it anywhere without the need to manually move/duplicate the python_module.so file. (Edit/Note : for now I don't want to publish my package on PiPy, just to be able to pip install locally the package)

Do you know how to do that ? I don't have a lot of knowledge about packaging, I've only done it once with a pyproject.toml and a setup.cfg.

The only examples I found use setup.py and are way more complicated. For example I must admit that I really don't understand how does this Pybind11 Cmake Example work.

I answer my own question, two months later, but I found the best example of what I wanted to do on this [cookiecutter](https://github.com/scikit-hep/cookie/) with scikit-build and PyBind11. [scikit-build](https://scikit-build.readthedocs.io/en/latest/) seems really promising to make the packaging of binary extensions an easier task. — Naomi, Jan 22 '23 at 15:10

score 0 · Answer 1 · answered Dec 02 '22 at 11:22

0

If I understand your question correctly, you want to be able to import your module without having to move the .so file manually.

Running python3 setup.py install should both compile your module via cmake and do the necessary linking. After that you should be able to import your module as usual.

answered Dec 02 '22 at 11:22

Yaroslav Biloshytskyi

43
4

I don't have a setup.py. I have a CMakeLists.txt creating a .so file ; the link I gave show a setup.py but then again I don't understand a line of it and how to adapt it to my project and I don't really find explanations online – Naomi Dec 05 '22 at 14:20
You can just copy the setup.py from that github project and change the setup() function at the end with information about your module. That script just does some dependency/platform checking and then runs the cmake commands. From what I understand using setup.py is the correct way of installing your module and is necessary if you want to upload your module to pypi for example. – Yaroslav Biloshytskyi Dec 05 '22 at 16:39

saell · Answer 2 · 2023-08-03T11:26:39.827

As described in another question, I installed the created library to the python<version>/site-packages directory:

find_package(Python3 COMPONENTS Interpreter Development REQUIRED)

...

pybind11_add_module( mylib MODULE
            mylib.cxx
            )

...

install(TARGETS mylib
        COMPONENT python
        LIBRARY DESTINATION "lib/python${Python3_VERSION_MAJOR}.${Python3_VERSION_MINOR}/site-packages"
        ARCHIVE DESTINATION "lib"
        RUNTIME DESTINATION "bin")

That way the python interpreter finds the library without additional manipulation of PYTHONPATH

The variables Python3_VERSION_MAJOR and Python3_VERSION_MINOR are provided by the `find_package' instruction

While this link may answer the question, it is better to include the essential parts of the answer here and provide the link for reference. Link-only answers can become invalid if the linked page changes. - [From Review](/review/late-answers/34676615) — Sebastian Wozny, Jul 14 '23 at 09:18

How to create a Python package from a PyBind11 module created with CMake

2 Answers2