Adding a minimal `pyproject.toml` silently breaks my build

Question

I am trying to use pybind11 and scikit-build in a minimal repository based on their cpp example repo. When trying to build it via pip install -e ., pip claims that the file was "Successfully installed", but didn't actually build anything (I suspect it never even ran the setup function inside setup.py). I can confirm it doesn't work given the error message ImportError: cannot import name 'example' from 'hello' when attempting to use the module.

Interestingly enough, the project works when I either

run python setup.py develop directly (thus bypassing pyproject.toml, pressumably). For this, I installed setuptools, scikit-build, cmake, pybind11, and ninja to their appropriate version through conda install X.
Deleting pyproject.toml, relying on pre-installed build-time dependencies.

I don't see how my pyproject.toml file would be problematic. Can anyone help me figure out what's wrong?

My project layout is

hello-cpp
├── hello
|    └── __init__.py (empty)
|    └── hello.cpp
├── CMakeLists.txt
├── pyproject.toml
├── setup.py

setup.py:

from skbuild import setup

setup(
name="hello-cpp",
    version="1.2.3",
    description="a minimal example package (cpp version)",
    author="Me",
    license="MIT",
    packages=["hello"],
    python_requires=">=3.7",
)

CMakeLists.txt:

cmake_minimum_required(VERSION 3.4...3.22)

project(hello)

find_package(pybind11 CONFIG REQUIRED)

pybind11_add_module(example hello/hello.cpp)
install(TARGETS example LIBRARY DESTINATION hello)

pyproject.toml (I'll note that the problem persists even without constraining versions here):

[build-system]
requires = ["setuptools==65.6.3", 
      "scikit-build==0.15.0", 
      "cmake>=3.18", 
      "pybind11==2.9.1",
      "ninja"
]
build-backend = "setuptools.build_meta"

src/hello.cpp:

#include <pybind11/pybind11.h>

int add(int i, int j) {
    return i + j;
}

PYBIND11_MODULE(example, m) {
    m.doc() = "pybind11 example plugin"; // optional module docstring

    m.def("add", &add, "A function that adds two numbers");
}

I'm trying this in Ubuntu 20.

EDIT: I am attempting to build this in a conda environment that has no packages other than python=3.10 (and whatever packages are bundled by default) installed. I've also tried it after conda installing the packages mentioned above.

The python script that fails with the ImportError upon adding pyproject.toml is

python -c "from hello import example"

Try `python -m pip install --editable .` https://snarky.ca/why-you-should-use-python-m-pip/ -- Are you using a Python virtual environment? Or a conda environment? — sinoroc, Mar 14 '23 at 10:17
I get the same behavior with `python -m pip install --editable .` than I do with `pip install -e .` I am using a conda environment — polortiz40, Mar 14 '23 at 17:28
Maybe edit your question to clarify that you are using conda environments. It might also be interesting to see which code leads to the *import error*, because the code doing the import itself might also be the reason why it fails. — sinoroc, Mar 14 '23 at 18:58
Sure, this is the output: `['__builtins__', '__cached__', '__doc__', '__file__', '__loader__', '__name__', '__package__', '__path__', '__spec__']` — polortiz40, Mar 14 '23 at 20:21
OK. Seems like `hello` can be imported, but there is nothing usable in it, or at least no `hello.example`. — sinoroc, Mar 14 '23 at 20:26

Henry Schreiner · Answer 1 · 2023-03-29T20:20:02.697

FYI, I copied your files into a directory and tried this:

$ virtualenv .venv
$ .venv/bin/pip install .
$ ls .venv/lib/python3.11/site-packages/hello
__init__.py                    __pycache__/                   example.cpython-311-darwin.so*

And it seems fine. Note that setuptools + CMake custom extension, scikit-build, and scikit-build-core all do not support editable installs (-e/--editable) since Setuptools 64. If you force the old implementation, it might sort-of work, but really isn't ideal still - you aren't getting much "editable" out of it. Scikit-build-core will have a proper editable installation mechanism soon (within a month or two).

Adding a minimal `pyproject.toml` silently breaks my build

1 Answers1