How to build a PEP384-compliant Python extension module and package it with correct ABI3 tag?

Question

mymodule.c starts as follows:

#define PY_SSIZE_T_CLEAN
#define Py_LIMITED_API 0x03070000 
#include "Python.h"

It builds with no errors and warnings. Yet, the resulting file is named

mymodulename.cpython-37m-x86_64-linux-gnu.so

Build command:

$ python setup.py build

When I issue instead

$ pip wheel .

the extension contained in the wheel is named the same, and consequently the wheel has no ABI3 tag either.

I expected a file-name with abi3 or alike, as set out in PEP 425.

My research so far was frustrating. I looked at PEPs 384 and 425, the docs and relevant tutorials on docs.python.org, in particular its C/API and distutils, the docs from PYPA, those on setuptools, wheel and pip - to no avail.

The file /modules/xxmodule.c contained in the current Python source distro is merely of historical interest.
What am I missing here?

Answer 1

It depends on the backend; not all build backends support PEP 384.

distutils

For example, when using pure distutils (so

from distutils.core import setup

setup(...)

), there's no way to change the extension suffix; this is why I had to ask the question Change Cython's naming rules for .so files some years ago. But you shouldn't be using pure distutils nowadays anyway, so this section is just for the sake of completeness.

setuptools

If you are using setuptools, you have to pass py_limited_api=True when creating Extension objects. Example:

from setuptools import setup, Extension


setup(
    ...,
    ext_modules=[
        # this extension will have the SOABI suffix, e.g.
        # cpython-37m-darwin or cpython-39-x86_64-linux-gnu etc.
        Extension("foo", ["foo.c"]),
        # this extension will have the `abi3.so` suffix
        Extension("bar", ["bar.c"], py_limited_api=True),
    ],
)

setuptools_rust

Bonus: If you are building extension modules from Rust code and happen to use setuptools_rust, building PEP 384-compliant extensions is now also possible since v0.11.4:

from setuptools import setup
from setuptools_rust import RustExtension


setup(
    ...,
    rust_extensions=[
        RustExtension("foo.rust", py_limited_api=True, features=["pyo3/abi3"]),
    ],
)

pip wheel: building the correct wheel tag

When I issue

$ pip wheel .

the extension contained in the wheel is named the same, and consequently the wheel has no ABI3 tag either.

The wheel tag is a somewhat different story; it is independent from the extension names it packages. To set the limited ABI, when building a wheel the usual way, one would issue

$ python setup.py bdist_wheel --py-limited-api=cp37

The resulting wheel would then have the name <pkg>-<ver>-cp37-abi3-<platform>.whl.

However, this won't work with pip wheel as you can't pass options to the bdist_wheel subcommand. You can thus persist it in the setup.cfg:

# setup.cfg
[bdist_wheel]
py_limited_api = cp37

When running pip wheel . now, bdist_wheel will pick up the option from the setup.cfg and assemble the correct wheel name.