46

I'm trying to put a Python project into a tarball using setuptools. The problem is that setuptools doesn't appear to like the way that the source tree was originally setup (not by me, I must add). Everything that I actually want to distribute is in the top-level directory, rather than in a subdirectory like the setuptools docs talk about.

The tree has a directory, tests, that I don't want to have in the released package. However, using exclude_package_data doesn't seem to actually do any excluding, and I'd like to work out what I've done wrong.

My setup.py looks like this, in relevant part:

setup(
  name="project",
  packages=[''],
  include_package_data=True,
  exclude_package_data={'': ['tests']},
  test_suite='nose.collector',
)
womble
  • 12,033
  • 5
  • 52
  • 66

7 Answers7

58

I have wasted several hours on the same problem, trying to exclude a module, I finally found that I had to remove the *.egg-info and build directories, that somehow retained the idea that the module had to be included.

mok0
  • 1,101
  • 1
  • 9
  • 8
  • 15
    Oh my... Thank you! Hours wasted here too. – Felix Jan 14 '20 at 10:15
  • 9
    Again, many, MANY thanks! I wasted all morning trying to fix this idea. This entire packaging and install if WAY over complicated and WAY too buggy. Just look at all the problems. But getting it fixed might require a tremendous effort in designing an entirely new system so we don't have to drag all this backward compatibility nonsense. YUCK! – Trailing Dots May 28 '20 at 18:53
  • 5
    In my case, removing only the `*.egg-info` directory was sufficient. – djvg Aug 24 '21 at 12:58
  • Wasted two hours on this, thanks so much for clarifying!! – rtindru Oct 07 '21 at 17:57
  • 3
    Removing the `egg-info` directory did it, god I was already on track to wasting another evening! – Javier Villa Jan 05 '22 at 19:07
  • 1
    Ditto, many thanks!! This is entirely not documented, from what I can tell. – Paco Nov 24 '22 at 02:35
  • WTF. That might explain why so many answers about building packages appear broken or as if they are doing strictly nothing. They might actually be working. You just don't see the results without first removing egg-info folder. Sigh.. – Eric Duminil Mar 10 '23 at 07:03
34

We use the following convention to exclude 'tests' from packages.

setup(
   name="project",
   packages=find_packages(exclude=("tests",)),
   include_package_data=True, 
   test_suite='nose.collector',
)

We also use MANIFEST.in to better control what include_package_data=True does.

ndmeiri
  • 4,979
  • 12
  • 37
  • 45
flexiondotorg
  • 341
  • 3
  • 2
  • 10
    Oddly enough, using `exclude=()` was not enough for me. Per http://stackoverflow.com/a/11669299/472876 , I had to use `recursive-exclude tests *` in my `MANIFEST.in` for success. – HeyWatchThis Aug 12 '14 at 16:28
  • 2
    @HeyWatchThis Did your `tests/` directory have a `__init__.py` file in it? Maybe that makes a difference? – code_dredd Feb 22 '19 at 00:19
  • 3
    Don't forget to clean-up your `build`, `dist` and `*.egg-info` directories. – Trevor Track May 30 '20 at 20:54
  • @code_dredd: if it was a regular folder and not a Python package you woudn't have to exclude it from finding PACKAGES in the first place. BTW, I have the same issue as HeyWatchThis. Python and it's stunted and misshapen distribution process... – z33k Mar 18 '22 at 10:21
21

This is what I found in setuptools manual:

from setuptools import setup, find_packages
...
packages = find_packages(exclude=["*.tests", "*.tests.*", "tests.*", "tests"]),

In my case the following was enough to exclude top-level tests directory:

packages = find_packages(exclude=["tests.*", "tests"]),
Enno Gröper
  • 4,391
  • 1
  • 27
  • 33
13

I have the following in my setup.py...

setup(name='pyfoo',
      version="1.0.2",
      description='Example for stack overflow',
      url='http://stackoverflow.com/',
      author='David Michael Pennington',
      author_email='mike /|at|\ pennington.net',
      license='GPL',
      platforms='any',
      keywords='Stack Overflow Example',
      entry_points = "",
      long_description=read('README.rst'),
      include_package_data=True,  # Checks MANIFEST.in for explicit rules
      #                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
      packages=find_packages(),
      use_2to3=True,
      zip_safe=False,
      setup_requires=["setuptools_hg"],

I had a doc/ directory that was getting massive due to the number of images I had in it; this meant that the size of my sdist was growing over 500kB. Originally I had this in my MANIFEST.in...

include LICENSE CHANGES README.rst requirements.txt
recursive-exclude * __pycache__
recursive-exclude * *.pyc
recursive-exclude * *.pyo
recursive-exclude * *.orig

The only thing I had to do to exclude my doc directory was this line at the bottom of MANIFEST.in...

prune doc*

Using prune doc* suddenly removed all my doc/ directory from the sdist tarball. So, it looks like you just need to use this in a MANIFEST.in file...

prune tests*
Mike Pennington
  • 41,899
  • 19
  • 136
  • 174
6

Ug, setuptools makes this really tricky :(

I don't know if this is what you want, but one project I work on uses a combination of two things:

from setuptools import setup, find_packages
...
packages = find_packages(exclude=['tests']),
data_files = os.walk(path_to_files),
David Wolever
  • 148,955
  • 89
  • 346
  • 502
5

An additional solution that worked in my case. Apparently: packages=setuptools.find_packages(exclude=["tests.*", "tests"]),

didn't work, but:

packages=setuptools.find_packages(exclude=["*tests.*", "*tests"]),

adding the star character at the beginning of the word did the trick.

Kotsuki
  • 61
  • 1
  • 3
1

For similar purpose, my collegue wrote setuptools-dummy package: http://github.com/ella/setuptools-dummy/tree/master

Take a look at setuptools_dummy, modify excludes to your needs and it should work. If not, open an issue ;)

Almad
  • 5,753
  • 7
  • 35
  • 53