add data files to python projects setup.py

Question

I have a project like this:

├── CHANGES.txt
├── LICENSE
├── MANIFEST.in
...
├── docs
│   └── index.rst
├── negar
│   ├── Negar.py
│   ├── Virastar.py
│   ├── Virastar.pyc
│   ├── __init__.py
│   ├── data
│   │   ├── __init__.py
│   │   └── untouchable.dat
│   ├── gui.py
│   ├── gui.pyc
│   ├── i18n
│   │   ├── fa_IR.qm
│   │   └── fa_IR.ts
│   └── negar.pro
├── setup.py
...

and inside that my file Virastar.py need some data from data.untouchable.dat. it works fine until I install the project with this setup.py:

setup(
    ...
    include_package_data=True,
    packages = find_packages() + ['negar'],
    package_dir={'negar': 'negar'},
    package_data={'negar': ['data/*.dat']},
    entry_points={
        'console_scripts': [
            'negar = negar.Negar:main',
        ],
    }, 
    ...  
)

after that when I start my program and when it needs that data file it return this error:

IOError: [Errno 2] No such file or directory: 'data/untochable.dat'

even in my egg-info sources I can't find any data file:

...
negar/Negar.py
negar/Virastar.py
negar/__init__.py
negar/gui.py
negar/data/__init__.py

have I missed something here?

Thank you all.

EDIT: Do I have to add any special thing in init.py?

and I have to add this: I used untouchable.dat just like this:

f = codecs.open('data/untouchable.dat', encoding="utf-8")

in python 2.7 they changed the way to include files to MANIFEST.in or something - I'm not sure and didn't use it but it could be a direction — zenpoy, Aug 30 '12 at 15:20
I'm in python 2.6! and for some reasons I can't upgrade now! — Shahin, Aug 30 '12 at 15:39

score 22 · Answer 1 · answered Mar 10 '13 at 14:19

22

I used data_files

data_files = [('', ['negar/data/untouchable.dat'])],

answered Mar 10 '13 at 14:19

dfrankow

20,191
41
152
214

1

For those wondering where the file will go: installation destination will be `sys.prefix`. Suggest replacing `''` with a dir-name to make some sense for the destination. – Jari Turkia Feb 23 '22 at 12:22
@JariTurkia Ah. So, are you suggesting `('negar/data', ['untouchable.dat'])`? – dfrankow Feb 23 '22 at 16:48
1

On the tuple, left side is destination, right side is source file. By having the destination as empty, during setup the file will be placed in the root of your Python's prefix. My suggestion is to have a directory like this: `[('negar/data', ['negar/data/untouchable.dat'])]` – Jari Turkia Feb 23 '22 at 18:08

score 13 · Accepted Answer · edited May 23 '17 at 12:10

13

The first problem is that I didn't import my data file into the package with MANIFEST.in file. I imported it like this:

include negar/data/*.dat

After that my data file already imported with my package install. but because I had mistakes in open my data files, python couldn't find it. this question helped me to find the right way Python Access Data in Package Subdirectory and now I use something like this:

import os
this_dir, this_filename = os.path.split(__file__)
DATA_PATH = os.path.join(this_dir, "data", "data.txt")
print open(DATA_PATH).read()

edited May 23 '17 at 12:10

Community

1
1

answered Aug 31 '12 at 05:56

Shahin

1,415
4
22
33

1

in which file did you add the lines `import os ...`? – Lucas May 24 '20 at 02:56

score 3 · Answer 3 · answered Aug 30 '12 at 07:38

3

Maybe try:

package_data={'negar/data': ['data/*.dat']},

answered Aug 30 '12 at 07:38

schacki

9,401
5
29
32

nyxgear · Answer 4 · 2022-09-08T13:13:34.463

A solution that does not require any of:

MANIFEST.in
include_package_data=True
package_dir={}
and neither the __init__.py file in the folder!

Having the project like:

├── CHANGES.txt
├── LICENSE
...
├── negar
│   ├── Negar.py
│   ├── Virastar.py
│   ├── Virastar.pyc
│   ├── __init__.py
│   ├── data
│   │   ├── __init__.py
│   │   └── untouchable.dat
│   ├── subfolder
│   │   ├── __init__.py
│   │   ├── data_NOT_included
│   │   │   └── garbage.toml
│   │   └── data_with_no_init_file
│   │       ├── config.ini
│   │       └── options.yml
│   ├── gui.py
│   ├── gui.pyc
│   ├── i18n
│   │   ├── fa_IR.qm
│   │   └── fa_IR.ts
│   └── negar.pro
├── setup.py
...

The setup file:

setup(
    ...
    packages = find_packages()
    package_data= {
        # all .dat files at any package depth
        '': ['**/*.dat'],

        # into the data folder (being into a module) but w/o the init file
        'negar.subfolder': [ '**/*.ini', '**/*.yml', ]
        
    },
    entry_points={
        'console_scripts': [
            'negar = negar.Negar:main',
        ],
    }, 
    ...
)

This worked great in my case and I avoided maintaining an additional MANIFEST.in file.

caveat: the "negar.subfolder" has to be a python module.

To the access the file:

from importlib.resources import files
config = files('negar.subfolder').joinpath('data_with_no_init_file').joinpath('config.ini').read_text()

from accessing-data-files-at-runtime

add data files to python projects setup.py

4 Answers4

Linked

Related