31

With the following package structure

.
├── my_package
│   └── __init__.py
├── setup.cfg
└── setup.py

Contents of setup.py

from setuptools import setup
setup()

Contents of setup.cfg

[metadata]
name = my_package
version = 0.1

[options]
packages = find:

I can build wheel or a source distribution for my_package like this

pip wheel --no-deps -w dist .
# generates file ./dist/my_package-0.1-py3-none-any.whl
python setup.py sdist
# generates file ./dist/my_package-0.1.tar.gz

But according to maintainer of setuptools, a declarative build configuration is ideal and using an imperative build is going to be a code smell. So we replace setup.py with pyproject.toml:

.
├── my_package
│   └── __init__.py
├── setup.cfg
└── pyproject.toml

Contents of pyproject.toml

[build-system]
build-backend = "setuptools.build_meta"
requires = ["setuptools", "wheel"]

And you can still build a wheel the same way as before, it works. But sdist doesn't work:

python: can't open file 'setup.py': [Errno 2] No such file or directory

So how should you actually build the .tar.gz file using setuptools? What's the user-facing tool to create sdist? I do not want to change the build backend. It looks like other packaging tools all write their own build entry points, but I thought the whole point of defining a declarative build system in the metadata was so that you didn't have to get hands-on with the build system, learning how each different packaging tool expects to be invoked or having to go into the interpreter and calling a Python API manually. But the PEP for build system requirements is over 2 years old now. Am I missing something obvious here?

How to build a source distribution without using setup.py file?

platypus
  • 1,128
  • 8
  • 15

3 Answers3

38

This is a somewhat controversial topic, and the answer for the moment is that there is no one single tool that everyone agrees is the "right way" to build source distributions, nor what that tool would be. You can see a long thread about it on the Python Packaging discourse.

I hesitate to give too much packaging advice in durable formats because the sands are always shifting, but as of November 2019, setup.py sdist is not deprecated, but it does have all the downsides that PEP 517 and PEP 518 were intended to fix - namely that you have to create the build environment yourself (and know about all the build dependencies), and it only works with setuptools/distutils and their equivalents.

It is not an "official" recommendation, but the current (Dec. 2020) best replacement for setup.py sdist and setup.py bdist_wheel is using pypa-build. Install once with

pip install build

and use as

python -m build --sdist --wheel

This builds source distribution and wheel at the same time. This is how I build my PEP 517-compatible packages.

This requires that your project have a pyproject.toml, and the pyproject.toml must have build-system.requires and build-system.build-backend keys, but it will work for any project with a PEP 517-compatible backend (including flit).

Other tools

Why not use flit or poetry or hatch? Those tools are all available for those who want to use them, but they are not an answer to this question. This question is asking about projects build with setuptools that use the declarative setup.cfg format. Neither flit nor poetry act as generic PEP 517 build front-ends, and so they only work as build commands for projects using their respective backends.

I am not familiar enough with hatch to say whether or not it can manage projects with backends other than setuptools, but (again, as of November 2019), it is not a PEP 517 frontend, and it will not work if you don't have a setup.py (it will raise the error "can't open file setup.py", and it will ignore your pyproject.toml file).

Al Sweigart
  • 11,566
  • 10
  • 64
  • 92
Paul
  • 10,381
  • 13
  • 48
  • 86
  • 1
    Why focus on `pep517.build` which is only meant as an experiment, a temporary crutch when there are productive tools such as flit, poetry, hatch, and probably even more? – sinoroc Nov 07 '19 at 20:43
  • 3
    Because it was a successful experiment in my estimation (I and many other PyPA people use it), as it has the right semantics for the job, and because it is the only general-purpose PEP 517 build front-end that I know. flit and poetry are vertically integrated in that they expect you to use their backend. hatch appears to do many other things. `pep517.build` is a simple tool built for exactly this purpose. – Paul Nov 07 '19 at 20:48
  • 1
    Ah right, good point. I was focusing on the build back-ends. And I actually thought `pep517.build` was one of them. But not at all, it's actually a build front-end. Also _hatch_ is not PEP517 ready as I see now. – sinoroc Nov 07 '19 at 21:01
  • Yes, perfect. I delete my answer. – sinoroc Nov 07 '19 at 21:05
  • According to the author themselves, this `pep517` package is not intended to be invoked directly: [_It's meant to be infrastructure for other tools like pip, not really a user-facing tool itself_](https://github.com/pypa/pep517/issues/54). So what is the "other tool" here, or what is the pip command to build your package? Without needing to manually select the correct interpreter and build environment, for example. – platypus Nov 08 '19 at 17:58
  • @platypus If you read the thread I linked, the answer is "there is no better tool right now". There is profound disagreement about what direction packaging tools should go in. The people who want an all-in-one tool have not built that tool, and most of the people who want a "simple unix-philosophy" tool already have `pep517.build` and haven't bothered to make their own wrapper. You do not need to select the correct build environment for `pep517.build`, you just need to specify an interpreter. – Paul Nov 08 '19 at 20:42
  • Thank you for this answer. The dearth of information on building an sdist from a project using pyproject.toml is enough to drive someone insane. AFAICT, `pep517` is more than an experiment, it's a library used by other projects to solve this problem: it's one of the vendored libs of pip. That said, it's true that `pep517.build` was not really meant to be used as a CLI long term, but it seems like the most straight-forward solution until `pip build` is implemented. Here is the issue for that: https://github.com/pypa/pip/issues/6041 – chadrik Nov 10 '19 at 23:53
  • There is hell, and then there is Python packaging! But thank you for the answer, if `python -m pep517.build` is good enough for Paul Ganssle then it is good enough for me. – platypus Nov 18 '19 at 15:34
  • 1
    Will the absence of setup.py affect editable installs? `pip install -e folder` – jerivas Apr 07 '20 at 17:10
  • @EduardoRivas As far as I know, as of today, if you want _editable_ installation you should keep the `setup.py`, that file could be kept as simple as `import setuptools; setuptools.setup()`, if all the information is already in `setup.cfg`. I believe the work is ongoing so that in the future the `setup.py` can be entirely deleted. – sinoroc Apr 14 '20 at 07:57
  • GGood point: `python -m build --sdist --wheel` – somenxavier Apr 10 '23 at 17:34
1

There's nothing "obvious" when it comes to Python packaging. Indeed, for the time being, at least if you are using distutils/setuptools it is necessary to create a (nearly) empty setup.py file, even if you are using a fully declarative setup.cfg:

#!/usr/bin/env python
from setuptools import setup
setup()

I also recommend chmod +x setup.py.

In this case you are just writing the "entry point" to the build system yourself, and setup() is just the main() function for it--but now all the arguments that were traditionally passed to setup() can be read from setup.cfg instead.

Now you can still use setup.py sdist if you want to make a source tarball:

./setup.py sdist

You could also try one of the alternative build systems that are enabled via pyproject.toml, such as Flit.

Iguananaut
  • 21,810
  • 5
  • 50
  • 63
  • 1
    Not sure why this is being downvoted; it's basically correct even if there are other solutions. – Iguananaut Nov 08 '19 at 09:42
  • 2
    The question title is "How to build a source distribution _without_ using setup.py file?" This answer seems to just demonstrate "here's how to recreate the same setup.py file that you've just deleted", which is not useful. – platypus Nov 08 '19 at 17:48
  • 1
    Yes, but that was based on a misunderstanding that writing a declarative `setup.cfg` means that a `setup.py` isn't necessary anymore to use setuptools, which isn't true. Just because the title of the question is misleading doesn't mean the answer is. They wrote in the body of the question "So how should you actually build the .tar.gz file *using setuptools*?" which this answers correctly. – Iguananaut Nov 09 '19 at 23:03
  • 2
    It actually is true. setuptools doesn't require a setup.py file if you are using PEP 517. – Paul Nov 19 '19 at 13:58
  • 2
    "if you are using PEP 517" Except most people aren't. It's still provisional, and is barely even mentioned in https://packaging.python.org . It's not something you would have unless you know to go looking for it. If you just want setuptools to work as it always used to this is correct. – Iguananaut Nov 19 '19 at 14:32
1

If you don't want to install a 3rd party tool and dont want to create a temporary setup.py, you also could use

python -c "import setuptools; setuptools.setup()" sdist
Gerhard Bräunlich
  • 19
  • 2