How to include a text file in a python installed package?

Question

I have created a python package that looks like this:

/command
    /command
        module.py
        __main__.py
    README.md
    setup.py
    file.txt

To install i run:

sudo python setup.py install

Now when i call

$ command

it shows this error:

FileNotFoundError: [Errno 2] No such file or directory: '../file.txt'

There are approximately the contents of modules setup.py, __main__.py and module.py

setup.py

import setuptools

setuptools.setup(
    name='command',
    ...
    entry_points={
        'console_scripts': [
            'command = command.__main__:main'
        ]
    }
)

__main__.py

from . import module

def main():
    module.run('arg')

module.py

import shutil

# copy file.txt from command project into the directory where it is running
def run(arg):
    shutil.copyfile('../file.txt', './file.txt')

After intalling this package via:

sudo python setup.py install

And calling the program at the command line

$ command

I get the error below

FileNotFoundError: [Errno 2] No such file or directory: '../file.txt'

How can I see and use a file that belongs to the installed package but I to use it in an environment where I am running that program?

Edit:

This a simplification of the problem you can download and test:

https://github.com/mctrjalloh/project_initializer

Is the file present in the installed package? If so, you can use `__file__` in a module to find out the absolute path of this module and calculate path to the data file from there — Michael Butscher, Nov 24 '18 at 00:14
@MichaelButscher i don't know if it is in there since it has been packaged in a .egg file that i can't open. — moctarjallo, Nov 24 '18 at 00:23
It seemed above that you tried to install the package already. So you should be able to check if the file is installed also. — Michael Butscher, Nov 24 '18 at 00:25
I have installed the package, the installation worked without any errors. it's when it come to using it that the error raises. but how can i check if file.txt is in that installed package if that's what you are asking.. — moctarjallo, Nov 24 '18 at 00:27
With some kind of file explorer or with the command line to list the files in the installation directory (and subdirectories). — Michael Butscher, Nov 24 '18 at 00:28
No, the whole package has been automatically packaged into a single .egg file — moctarjallo, Nov 24 '18 at 00:30
Eggs are actually zip-files. You can use a zip file viewer to check. See also https://stackoverflow.com/questions/3655352/how-to-access-files-inside-a-python-egg-file — Michael Butscher, Nov 24 '18 at 00:33
Let us [continue this discussion in chat](https://chat.stackoverflow.com/rooms/184166/discussion-between-mctrjalloh-and-michael-butscher). — moctarjallo, Nov 24 '18 at 01:13

moctarjallo · Accepted Answer · 2018-11-26T03:39:50.763

So after a lot research i found the solution to this and how it works also. It's a little bit confusing, none of the other stackoverflow answers were really that explanatory. I want to try it here:

I have made a sample project for that only purpose to demostrate and test the solution. I have come up with two solutions: one using the data_files argument of the setup() function and the other using the package_data argument which i preferred the most.

Here is the link to the github repo you can download and test

To use it after installation run

proj init <some-name>

But to be brief there are the most important modules for each method.

USING data_files= ARGUMENT METHOD:

Project structure:

project_initializer
    project_initializer
        __init__.py
        __main__.py
        init.py
    README.md
    setup.py

in setup.py

import setuptools
import os
import sys


PROJECT_NAME = "project_initializer"
DATA_DIR = os.path.join(
    sys.prefix, "local/lib/python3.6/dist-packages", PROJECT_NAME)


setuptools.setup(
    name='project_initializer',
    version='0.1.0',
    packages=setuptools.find_packages(),
    install_requires=[
        'docopt'
    ],
    data_files=[         # is the important part
        (DATA_DIR, [
            "README.md",
            ".gitignore"
        ])               
    ],
    entry_points={
        'console_scripts': [
            'proj = project_initializer.__main__:main'
        ]
    }
)

in init.py

import subprocess
import os
import shutil
import sys

"""Create a new project and initialize it with a .gitignore file
@params project: name of a project to be initialized
effects:
    /project
        README.md
    README.md in the created project directory must be the same as the README.md in THIS directory 
"""

PROJECT_NAME = "project_initializer"
DATA_DIR = os.path.join(
    sys.prefix, "local/lib/python3.6/dist-packages", PROJECT_NAME)


def run(project):
    os.mkdir(project)
    shutil.copyfile(os.path.join(DATA_DIR, "README.md"),
                    f"{project}/README.md")  # problem solved


if __name__ == '__main__':
    run("hello-world")

USING package_data= ARGUMENT METHOD (which i preferred)

Project structure:

project_initializer
    project_initializer
        data/
            README.md  # the file we want to copy
        __init__.py
        __main__.py
        init.py
    README.md
    setup.py

in setup.py

import setuptools


setuptools.setup(
    name='project_initializer',
    version='0.1.0',
    packages=setuptools.find_packages(),
    package_dir={'project_initializer': 'project_initializer'}, # are the ... 
    package_data={'project_initializer': [ # ... important parameters
        'data/README.md', 'data/.gitignore']},
    install_requires=[
        'docopt'
    ],
    entry_points={
        'console_scripts': [
            'proj = project_initializer.__main__:main'
        ]
    }
)

in init.py

import subprocess
import os
import shutil
import sys

PROJECT_DIR = os.path.dirname(__file__)

"""Create a new project and initialize it with a .gitignore file
@params project: name of a project to be initialized
effects:
    /project
        .gitignore
    .gitignore in the created project directory must be the same as the gitignore in THIS directory 
"""


def run(project):
    os.mkdir(project)
    shutil.copyfile(os.path.join(PROJECT_DIR, 'data/README.md'),
                    f"{project}/README.md")  # problem solved


if __name__ == '__main__':
    run("hello-world")

The reason why i prefer this last method is because you have not to import anything in the setup.py module which could be presumably a bad practice. I guess nothing should be imported in the setup.py file since it is an external file to the main package.

For more detailed explanation of what are the differences between the two arguments check out the python docs

Using data_files= argument

Using package_data= argument

score -2 · Answer 2 · answered Nov 24 '18 at 00:31

-2

Only python files are included in the package by default.

To include more add MANIFEST.in and list the file. For example.

Here is a comprehensive tutorial

answered Nov 24 '18 at 00:31

Rich Tier

9,021
10
48
71

MANIFEST.in is respected by diatutils when you build the package via `python setup.py bdist_wheel` – Rich Tier Nov 24 '18 at 00:49

How to include a text file in a python installed package?

2 Answers2

Linked