Making a python package for PIP with some data_files

Question

I'm doing a project with this layout:

project/
    bin/
        my_bin.py
    CHANGES.txt
    docs/
    LICENSE.txt
    README.txt
    MANIFEST.in
    setup.py
    project/
        __init__.py
        some_thing.py
        default_data.json
        other_datas/
            default/
                other_default_datas.json

And the problem is that when I install this using pip, it puts the "default_data.json" and the "other_datas" folder not in the same place as the rest of the app.

How am I supposed to do to make them be in the same place?

They end up on "/home/user/.virtualenvs/proj-env/project"

instead of "/home/user/.virtualenvs/proj-env/lib/python2.6/site-packages/project"

In the setup.py I'm doing it like this:

inside_dir = 'project'
data_folder= os.path.join(inside_dir,'other_datas')

data_files = [(inside_dir, [os.path.join(inside_dir,'default_data.json')])]
for dirpath, dirnames, filenames in os.walk(data_folder):
    data_files.append([dirpath, [os.path.join(dirpath, f) for f in filenames]])

score 7 · Accepted Answer · edited Jun 20 '20 at 09:12

7

From https://docs.python.org/3.4/distutils/setupscript.html#installing-additional-files:

If directory is a relative path, it is interpreted relative to the installation prefix (Python’s sys.prefix for pure-Python packages, sys.exec_prefix for packages that contain extension modules).

Each file name in files is interpreted relative to the setup.py script at the top of the package source distribution.

So the described behavior is simply how data_files work.

If you want to include the data files within your package you need to use package_data instead:

package_data={'project': ['default_data.json', 'other_datas/default/*.json']}

edited Jun 20 '20 at 09:12

Community

1
1

answered Sep 26 '14 at 09:21

tobib

2,114
4
21
35

6

It is impossible to use package_data if the file is outside the package root (which can happen e.g. if your package is inside 'src', but you want to add in a file outside of it). Isn't there a way to use data_files to go into the package? – E. T. Mar 08 '19 at 13:52

score 0 · Answer 2 · answered Feb 25 '15 at 12:14

0

Take a look at this package https://pypi.python.org/pypi/datafolder. It makes it easy to install and use (data files: *.conf, *.ini *.db, ...) by your package and by the user.

answered Feb 25 '15 at 12:14

AlexAtStack

366
2
7

1

This package messed up my build and wasted my time. Maybe just a little change here and there could fix the issue, but cannot recommend using it to others. – Vahid S. Bokharaie Aug 14 '19 at 15:24

score -1 · Answer 3 · answered Oct 30 '11 at 02:39

-1

Change your MANIFEST.in to include those .json.

It is probably gonna work:

recursive-include project/ *.json

answered Oct 30 '11 at 02:39

Hugo Lopes Tavares

28,528
5
47
45

1

There is already that in the manifest, and the files are going, but in the wrong folder. – Arruda Oct 31 '11 at 13:40

Making a python package for PIP with some data_files

3 Answers3