setuptools: adding additional files outside package

Question

I have a python application that has a fixed layout which I cannot change. I would like to wrap it up using setuptools, e.g. write a setup.py script.

Using the official documentation, I was able to write a first template. However, the application in question uses a lot of additional data files that are not explicitly part of any package. Here's an example source tree:

somepackage
   __init__.py
   something.py
   data.txt
additionalstuff
   moredata.txt
INFO.txt

Here's the trouble: The code in something.py reads the files moredata.txt and INFO.txt. For the former, I can monkeypatch the problem by adding an empty additionalstuff/__init__.py file to promote additionalstuff to a package and have it picked up by setuptools. But how could I possibly add INFO.txt to my .egg?

Edit

The proposed solutions using something along the lines of

package_data = { '' : ['moredata.txt','INFO.txt']}

does not work for me because the files moredata and INFO.txt do not belong to a package, but are part of a separate folder that is only part of the module as a whole, not of any individual package. As explained above, this could be fixed in the case of moredata.txt by adding a __init__.py file to additionpythonalstuff, thus promoting it to a package. However, this is not an elegant solution and does not work at all for INFO.txt, which lives in the top-level directory.

Solution

Based on the accepted answer, here's the solution

This is the setup.py:

from setuptools import setup, find_packages

setup(
    name='mytest',
    version='1.0.0',
    description='A sample Python project',
    author='Test',
    zip_safe=False,
    author_email='test@test.com',
    keywords='test',
    packages=find_packages(),
    package_data={'': ['INFO.txt', 'moredata.txt'],
                  'somepackage':['data.txt']},
    data_files=[('.',['INFO.txt']),
                ('additionalstuff',['additionalstuff/moredata.txt'])],
    include_package_data=True,
)

And this is the MANIFEST.in:

include INFO.txt
graft additionalstuff
include somepackage/*.txt

possible duplicate of [Including non-Python files with setup.py](http://stackoverflow.com/questions/1612733/including-non-python-files-with-setup-py) — pacholik, Sep 16 '15 at 13:14
It is not a duplicate. In the given question, the accepted solution is to use `package_data`. This I have already done. Please take note that the given solution to use `''` (empty string) does not work for me since it explicitly refers to "all packages", which does not apply to me, since the file I want to add does not belong to *any* package, which is precisely the problem I'm facing. — carsten, Sep 16 '15 at 13:39
How is INFO.txt referenced from inside somepackage. I tried this but now the relative paths between somepackage and INFO.txt has changed. — Omar, Jun 28 '22 at 22:08
See [data-files-support in setuptools documentation](https://setuptools.pypa.io/en/latest/userguide/datafiles.html#data-files-support). — djvg, Aug 04 '22 at 14:31

score 10 · Accepted Answer · answered Sep 16 '15 at 14:00

10

There is also data_files

data_files=[("yourdir",
             ["additionalstuff/moredata.txt", "INFO.txt"])],

Have a think about where you want to put those files. More info in the docs.

answered Sep 16 '15 at 14:00

pacholik

8,607
9
43
55

Will `yourdir` be relative to the dir where package gets installed? – mad_ Jan 08 '20 at 19:30
Yes. From docs: *If directory is a relative path, it is interpreted relative to the installation prefix*. – pacholik Jan 09 '20 at 08:41
1

That was my understanding too. but it is just taking the root prefix and is not installing in the intended package I want it to. Maybe I am doing something wrong. If you know how to do this would you mind posting an example or answering this similar question https://stackoverflow.com/questions/59653195/add-folders-and-subfolders-outside-of-main-module-using-setuptoolssetup-py/59662834#59662834 – mad_ Jan 09 '20 at 14:51
@mad_ I have experienced the same. On some platforms it will place it correctly, on some it won't. It is better to use package_data and move files to the package beforehand. – Thomas Wagenaar Jan 07 '23 at 15:47

score 1 · Answer 2 · answered Sep 16 '15 at 13:19

1

You need to use package_data. Put your setup.py in the root directory, then you just need to:

package_data={'': [
    'somepackage/*.txt',
    'additionalstuff/*.txt',
    '*.txt',
]

answered Sep 16 '15 at 13:19

Daniele Pantaleone

2,657
22
33

4

This response relies on the feature of `''` to refer to "all packages". However, the file `INFO.txt` does not belong to *any* package and is thus not picked up. – carsten Sep 16 '15 at 13:41
It does work for me. I have the exact same directory tree configuration as you do and in the root of my project I just have a README.md and setup.py. The README.md is correctly included using package_data. – Daniele Pantaleone Sep 16 '15 at 13:48
This surprises me. I have created a small testcase and again observe the same problem. You seem to do something differently from what I am doing. Could you provide a link to the mentioned project so I can investigate the architecture? – carsten Sep 16 '15 at 14:10
1

Sadly no, because it's in a private repository on github. However pacholik answer should be more correct than mine. – Daniele Pantaleone Sep 16 '15 at 14:12
2

This does not work for me. `data_files` fixes it but it's deprecated. What is the standard solution? Here is my file https://github.com/ilius/pyglossary/blob/master/setup.py – saeedgnu Oct 20 '20 at 08:25

setuptools: adding additional files outside package

Edit

Solution

2 Answers2

Linked