setuptools troubles -- excluding packages, including data files

Question

I'm fairly new to setuptools. I've seen a few similar questions and it drives a little bit insane that I've seemed to follow advice I saw but setuptools still does something different than what I want.

Here is the structure of my project:

.
..
package1/
    __init__.py
    abc.py
    ...
tests/
    __init__.py
    test_package1.py
LICENSE
README.md
RELEASE
setup.py

And here is the contents of my setup.py:

#!/usr/bin/env python
import os
#from distutils.core import setup
from setuptools import setup, find_packages

setup(
    name='package1',
    version='1.1',
    test_suite="tests",
    packages=find_packages(exclude=['tests']),    
    include_package_data=True,
    package_data = {
        '': ['LICENSE', 'README.md5', 'RELEASE']
    },   
)

Also, in my manifest file I have:

include LICENSE
include RELEASE
include README.md

I build the tar with:

python setup.py sdist

I want to:

Exclude tests directory from the source distribution;
Have LICENSE, README.md, RELEASE files in the site-packages directory, either at the top level, or inside the package1 directory (at this point I will agree to either).

Instead, here's what happens:

tests directory remains to be in the created tar archive and gets installed to the site-packages;
Files are copied to the archive, but do not get installed to the site-packaged directory of the package.

I am out of ideas, can someone explain to me what I am doing wrong and how to fix it?

why python doesn't love us... It is a very simple and common task: _ignore several files, and include several uncommon files to package_. Why is it so hard to do? — maxkoryukov, Aug 23 '17 at 18:46

score 28 · Answer 1 · edited Mar 21 '19 at 13:41

28

find_packages uses fnmatchcase for its exclude filtering. You can test if your exclusion pattern matches a package name as follows:

>>> from fnmatch import fnmatchcase
>>> fnmatchcase('my.package.name.tests', 'tests')
False

Assuming all the tests in your project live in package names ending in tests or subpackages of those packages, the following should suffice to exclude all the test code:

setup(
    name='package1',
    version='1.1',
    packages=find_packages(exclude=['tests', '*.tests', '*.tests.*']),    
)

To also exclude the tests folder from source distributions, add the following to MANIFEST.in:

recursive-exclude tests *

edited Mar 21 '19 at 13:41

phoenix

7,988
6
39
45

answered Oct 09 '14 at 21:20

oby1

1,302
1
13
20

This is the best answer to the question – Greg Oct 21 '14 at 05:42
3

I've struggled a lot with the syntax for `find_packages()` and a useful trick I've found is to `import setuptools` in a python console in the folder where your `setup.py` resides; and then do things like this to see what packages it will list in response to the file-matching patterns you choose: `print(setuptools.find_packages(where='YourSourceFolder', exclude=['tests', 'tests.*']))` – snark Aug 19 '20 at 14:03
If all the tests live in a subpackage, i.e. in a subdirectory under the main package directory, then it is apparently sufficient to specify `packages=find_packages(exclude=['*.tests', '*.tests.*']),` , and no `MANIFEST.in` file entry is needed. At least this is what my experiment under Python 3.7 told me. – András Aszódi Nov 05 '20 at 16:05
@snark: thx man, good piece of advice – z33k Mar 18 '22 at 10:36

score 24 · Accepted Answer · answered Jul 26 '12 at 12:27

You should create a new file called MANIFEST.in in the root level of your package, then follow these instructions:

To control which files end up in your tar file, create a new file called MANIFEST.in in the root level of your package. For example, you can exclude whole directories from your distribution, using recursive-exclude in the MANIFEST.in file. In your case, you need your MANIFEST.in file to contain:
```
recursive-exclude tests *
```
It's not common to include README and other files in the site-packages directory, but if you really want to, then go inside package1 and create symbolic links to the files you want to include:
```
cd package1
ln -s ../LICENSE
ln -s ../README.md
ln -s ../RELEASE
```
Then change the following line in your setup.py:
```
package_data = {
    '': ['LICENSE', 'README.md', 'RELEASE']
```
to:
```
package_data = {
    'package1': ['LICENSE', 'README.md', 'RELEASE']
```

score 6 · Answer 3 · answered Feb 06 '19 at 17:20

6

I was trying everything and nothing seemed to work, until I deleted the build directory (after seeing another answer mentioning *.egg-info/ directory) and it finally worked. python setup.py clean --all should also do the job.

answered Feb 06 '19 at 17:20

AXO

8,198
6
62
63

1

Thank you, this was the cause of the issues I was having. The setup.py clean --all did not work and clean everything for me. – robmsmt May 29 '21 at 19:32

score 0 · Answer 4 · answered Sep 20 '17 at 11:33

0

MANIFEST.in would take care of it.

prune tests/
include LICENSE  README.md RELEASE

if you have static files to be added, add them with

...
recursive-include package1/static *

answered Sep 20 '17 at 11:33

All Іѕ Vаиітy

24,861
16
87
111

score 0 · Answer 5 · edited Mar 16 '17 at 20:41

0

if your structure is true:

the tests folder is not a package (it doesn't have the init.py file), so find_package don't exclude it.
include_package_data=True mean that all versionned files will be included if not explicitly excluded .

so : try an exclude tests/* in your MANIFEST.in

I Hope it had helped

edited Mar 16 '17 at 20:41

Jérôme

13,328
7
56
106

answered Dec 19 '11 at 10:49

ornoone

651
5
11

2

I do have __init__.py in tests, actually, just did not show it in the source. In fact, if I print find_packages() result, it does only return package1. But the tests folder is still included. – 0x4B1D Dec 19 '11 at 15:25

setuptools troubles -- excluding packages, including data files

5 Answers5

Linked