How to access text file in project root from python package under `src/` directory

Question

I want my package's version number to live in a single place where everything that needs it can refer to it.

I found several suggestions in this Python guide to Single Sourcing the Package Version and decided to try #4, storing it in a simple text file in my project root named VERSION.

Here's a shortened version of my project's directory tree (you can see the the full project on GitHub):

.
├── MANIFEST.in
├── README.md
├── setup.py
├── VERSION
├── src/
│   └── fluidspaces/
│       ├── __init__.py
│       ├── __main__.py
│       ├── i3_commands.py
│       ├── rofi_commands.py
│       ├── workspace.py
│       └── workspaces.py
└── tests/
    ├── test_workspace.py
    └── test_workspaces.py

Since VERSION and setup.py are siblings, it's very easy to read the version file inside the setup script and do whatever I want with it.

But VERSION and src/fluidspaces/__main__.py aren't siblings and the main module doesn't know the project root's path, so I can't use this approach.

The guide had this reminder:

Warning: With this approach you must make sure that the VERSION file is included in all your source and binary distributions (e.g. add include VERSION to your MANIFEST.in).

That seemed reasonable - instead of package modules needing the project root path, the version file could be copied into the package at build time for easy access - but I added that line to the manifest and the version file still doesn't seem to be showing up in the build anywhere.

To build, I'm running pip install -U . from the project root and inside a virtualenv. Here are the folders that get created in <virtualenv>/lib/python3.6/site-packages as a result:

fluidspaces/
├── i3_commands.py
├── __init__.py
├── __main__.py
├── __pycache__/  # contents snipped
├── rofi_commands.py
├── workspace.py
└── workspaces.py
fluidspaces-0.1.0-py3.6.egg-info/
├── dependency_links.txt
├── entry_points.txt
├── installed-files.txt
├── PKG-INFO
├── SOURCES.txt
└── top_level.txt

More of my configuration files:

MANIFEST.in:

include README.md
include VERSION
graft src
prune tests

setup.py:

#!/usr/bin/env python3

from setuptools import setup, find_packages


def readme():
    '''Get long description from readme file'''
    with open('README.md') as f:
        return f.read()


def version():
    '''Get version from version file'''
    with open('VERSION') as f:
        return f.read().strip()


setup(
    name='fluidspaces',
    version=version(),
    description='Navigate i3wm named containers',
    long_description=readme(),
    author='Peter Henry',
    author_email='me@peterhenry.net',
    url='https://github.com/mosbasik/fluidspaces',
    license='MIT',
    classifiers=[
      'Development Status :: 3 - Alpha',
      'Programming Language :: Python :: 3.6',
    ],
    packages=find_packages('src'),
    include_package_data=True,
    package_dir={'': 'src'},
    package_data={'': ['VERSION']},
    setup_requires=[
        'pytest-runner',
    ],
    tests_require=[
        'pytest',
    ],
    entry_points={
        'console_scripts': [
            'fluidspaces = fluidspaces.__main__:main',
        ],
    },
    python_requires='~=3.6',
)

I found this SO question Any python function to get “data_files” root directory? that makes me think the pkg_resources library is the answer to my problems, but I've not been able to figure out how to use it in my situation.

I've been having trouble because most examples I've found have python packages directly in the project root instead of isolated in a src/ directory. I'm using a src/ directory because of recommendations like these:

• PyTest: Good Practices: Tests Outside Application Code
• Ionel Cristian Mărieș - Packaging a Python Library
• Hynek Schlawack - Testing and Packaging

Other knobs I've found and tried twisting a little are the package_data, include_package_data, and data_files kwargs for setup(). Don't know how relevent they are. Seems like there's some interplay between things declared with these and things declared in the manifest, but I'm not sure about the details.

Answer 1

Chatted with some people in the #python IRC channel on Freenode about this issue. I learned:

• pkg_resources was probably how I should to do what I was asking for, but it would require putting the version file in the package directory instead of the project root.
• In setup.py I could read in such a version file from the package directory without importing the package itself (a no-no for a few reasons) but it would require hard-coding the path from the root to the package, which I wanted to avoid.

Eventually I decided to use the setuptools_scm package to get version information from my git tags instead of from a file in my repo (someone else was doing that with their package and their arguments were convincing).

As a result, I got my version number in setup.py very easily:

setup.py:

from setuptools import setup, find_packages

def readme():
    '''Get long description from readme file'''
    with open('README.md') as f:
        return f.read()

setup(
    name='fluidspaces',
    use_scm_version=True,  # use this instead of version
    description='Navigate i3wm named containers',
    long_description=readme(),
    author='Peter Henry',
    author_email='me@peterhenry.net',
    url='https://github.com/mosbasik/fluidspaces',
    license='MIT',
    classifiers=[
      'Development Status :: 3 - Alpha',
      'Programming Language :: Python :: 3.6',
    ],
    packages=find_packages('src'),
    package_dir={'': 'src'},
    setup_requires=[
        'pytest-runner',
        'setuptools_scm',  # require package for setup
    ],
    tests_require=[
        'pytest',
    ],
    entry_points={
        'console_scripts': [
            'fluidspaces = fluidspaces.__main__:main',
        ],
    },
    python_requires='~=3.6',
)

but I ended up having to have a hard-coded path indicating what the project root should be with respect to the package code, which is kind of what I had been avoiding before. I think this issue on the setuptools_scm GitHub repo might be why this is is necessary.

src/fluidspaces/__main__.py:

import argparse
from setuptools_scm import get_version  # import this function

def main(args=None):
    # set up command line argument parsing
    parser = argparse.ArgumentParser()
    parser.add_argument('-V', '--version',
                        action='version',
                        version=get_version(root='../..', relative_to=__file__))  # and call it here

Answer 2

For folks still looking for the answer to this, below is my attempt at following variety #4 of the guide to Single Sourcing the Package Version. It's worth noting WHY you might choose this solutions when there are other simpler ones. As the link notes, this approach is useful when you have external tools that might also want to easily check the version (e.g. CI/CD tools).

File tree

myproj
├── MANIFEST.in
├── myproj
│   ├── VERSION
│   └── __init__.py
└── setup.py

myproj/VERSION

1.4.2

MANIFEST.in

include myproj/VERSION

setup.py

with open('myproj/VERSION') as version_file:
    version = version_file.read().strip()

setup(
    ...
    version=version,
    ...
    include_package_data=True,  # needed for the VERSION file
    ...
)

myproj/__init__.py

import pkgutil

__name__ = 'myproj'
__version__ = pkgutil.get_data(__name__, 'VERSION').decode()

It's worth noting that setting configuration in setup.cfg is a nice, clean alternative to including everything in the setup.py setup function. Instead of reading version in setup.py, and then including in the function, you could do the following:

setup.cfg

[metadata]
name = my_package
version = attr: myproj.VERSION

In the full example I chose to leave everything in setup.py for the ease of one less file and uncertainty about whether or not potential whitespace around the version in the VERSION file would be stripped by the cfg solution.