including python file from project root in setup.py build

Question

I am trying to include a python file in the build/lib directory created when running

python setup.py install

In particular, I would like to include a simple configuration file ('definitions.py') that defines a ROOT_DIR variable, which is then used by subpackages. The 'definitions.py' file contains:

import os
ROOT_DIR  = os.path.dirname(os.path.abspath(__file__))

My goal is to have configuration files within each subpackage ('config.py') call ROOT_DIR to build their own absolute paths:

from definitions import ROOT_DIR
PACKAGE_DIR = os.path.join(ROOT_DIR, 'package1/')

The idea is drawn from this stackoverflow answer: https://stackoverflow.com/a/25389715.
However, this 'definitions.py' file never shows up in the build directory when running 'setup.py install'.

Here is the directory structure of the project:

project
|
├── setup.py
|
├── definitions.py
|
├── package1
|   ├── __init__.py
|   ├── config.py
|   └── ...
|
├── package2
|   ├── __init__.py
|   └── ...
└── ...

My multiple attempts have failed (trying, e.g. the suggestions offered in https://stackoverflow.com/a/11848281). As far as I can tell, it's because definitions.py is in the top-level of my project structure (which lacks an __init__.py file).

I have tried:

1) ...using the 'package-data' variable in setuptools.setup()

package_data={'package': ['./definitions.py']}

but definitions.py does not show up in the build
(I think because definitions.py is not in a 'package' that has an __init__.py?).

2) ...using a MANIFEST.in file, but this also does not work
(I think because MANIFEST does not work with .py files?)

My question:
Is there a way to include definitions.py in the build directory?
Or, is there a better way to provide access to absolute paths built from the top-level directory for multiple sub-packages?

Answer 1

If you are looking for a way to access a non-python data file in the installed module like in the question you've linked (a configuration file in the top-level package that should be accessible in subpackages), use pkg_resources machinery instead of inventing a custom path resolution. An example project structure:

project
├── setup.py
└── root
    ├── __init__.py
    ├── config.txt
    ├── sub1
    │   └── __init__.py
    └── sub2
        └── __init__.py

setup.py:

from setuptools import setup

setup(
    name='myproj',
    ...,
    packages=['root', 'root.sub1', 'root.sub2'],  # or setuptools.find_packages()
    package_data={'root': ['config.txt']}
)

Update:

As pointed out by wim in the comments, there's now a backport for importlib.resources (which is only available in Python 3.7 and onwards) - importlib_resources, which offers a modern resource machinery that utilizes pathlib:

# access the filepath
importlib_resources.path('root', 'config.txt')

# access the contents as string
importlib_resources.read_text('root', 'config.txt')

# access the contents as file-like object
importlib_resources.open_binary('root', 'config.txt')

Original answer

Using pkg_resources, you can access the root/config.txt from any spot of your package without having to perform any path resolution at all:

import pkg_resources

# access the filepath:
filepath = pkg_resources.resource_filename('root', 'config.txt')

# access the contents as string:
contents = pkg_resources.resource_string('root', 'config.txt')

# access the contents as file-like object:
contents = pkg_resources.resource_stream('root', 'config.txt')

etc.