Make ``pip install -e .`` build cython extensions with pyproject.toml

Question

With the move to the new pyproject.toml system, I was wondering whether there was a way to install packages in editable mode while compiling extensions (which pip install -e . does not do).

So I want pip to:

• run the build_ext I configured for Cython and generate my .so files
• put them in the local folder
• do the rest of the normal editable install

I found some mentions of build_wheel_for_editable on the pip documentation but I could not find any actual example of where this hook should be implemented and what it should look like. (to be honest, I'm not even completely sure this is what I'm looking for)

So would anyone know how to do that? I'd also happy about any additional explanation as to why pip install . runs build_ext but the editable command does not.

---

Details:

I don't have a setup.py file anymore; the pyproject.toml uses setuptools and contains

[build-system]
requires = ["setuptools>=61.0", "numpy>=1.17", "cython>=0.18"]
build-backend = "setuptools.build_meta"

[tool.setuptools]
package-dir = {"" = "."}

[tool.setuptools.packages]
find = {}

[tool.setuptools.cmdclass]
build_ext = "_custom_build.build_ext"

The custom build_ext looks like

from setuptools import Extension
from setuptools.command.build_ext import build_ext as _build_ext

from Cython.Build import cythonize

class build_ext(_build_ext):

    def initialize_options(self):
        super().initialize_options()
        if self.distribution.ext_modules is None:
            self.distribution.ext_modules = []
        extensions = Extension(...)
        self.distribution.ext_modules.extend(cythonize(extensions))

    def build_extensions(self):
        ...
        super().build_extensions()

It builds a .pyx into .cpp, then adds it with another cpp into a .so.

Answer 1

I created a module that looks like this:

$  tree .   
.
├── pyproject.toml
├── setup.py
└── test
    └── helloworld.pyx

1 directory, 3 files

My pyproject.toml looks like:

[build-system]
requires = ["setuptools>=61.0", "numpy>=1.17", "cython>=0.18"]
build-backend = "setuptools.build_meta"

[tool.setuptools]
py-modules = ["test"]

[project]
name        = "test"
version     = "0.0.1"%    

My setup.py:

from setuptools import setup
from Cython.Build import cythonize

setup(ext_modules=cythonize("test/helloworld.pyx"))

And helloworld.pyx just contains print("Hello world").

When I do pip install -e ., it builds the cython file as expected.

If you really don't want to have a setup.py at all, I think you'll need to override build_py instead of build_ext, but IMO just having the simple setup.py file isn't a big deal.