How to tell pbr to include non-code files in package

Question

I only run into this problem when I build in a python:alpine image. Reproducing it is a bit of a pain, but these are the steps:

Docker container setup:

$ docker run -it python:3.7-rc-alpine /bin/ash
$ pip install pbr

Small package setup, including non-python files

test
├── .git
├── setup.cfg
├── setup.py
└── src
    └── test
        ├── __init__.py
        ├── test.yml
        └── sub_test
            ├── __init__.py
            └── test.yml

setup.py:

from setuptools import find_packages, setup
setup(
    setup_requires=['pbr'],
    pbr=True,
    package_dir={'': 'src'},
    packages=find_packages(where='src'),
)

setup.cfg:

[metadata]
name = test

All other files are empty. I copy them to the container with docker cp test <docker_container>:/test.

Back in the container I now try to build the package with cd test; pip wheel -w wheel ., the test.yml in test/src/test will be included in it, but the one in test/src/test/sub_test won't.

I have no clue why this happens, since the (pitifully sparse, and imo quite confusing) documentation of pbr on that matter states that

Just like AUTHORS and ChangeLog, why keep a list of files you wish to include when you can find many of these in git. MANIFEST.in generation ensures almost all files stored in git, with the exception of .gitignore, .gitreview and .pyc files, are automatically included in your distribution.

I could not find a pbr-parameter that lets me explicitly include some file or file type, which I expected to exist.

Creating a MANIFEST.in with import src/test/sub_test.test.yml actually solves this problem, but I'd rather understand and avoid this behavior all together instead.

Answer 1

pbr needs git in order to correctly compile its files-to-include list, so the problem can be solved by installing git into the build environment before building the package. With an alpine image, that would be apk add --no-cache git.

---

pbr uses the .git file to figure out which files should be part of the package and which ones shouldn't. The short version is that it grabs the intersection of the file list from the packages parameter in the setup-call in setup.py and everything that is commited or staged in the currently checked out git branch.

So if the project came with no .git file, you'd need to additionally execute git init; git add src as well.

The reason for the 'bug' is that pbr silently assumes that all .py files should be added regardles of them being commited or not, which makes the actual problem harder to identify. It will also only throw an error if it can't find a .git file, and not if it is there but it can't get any info from it because git isn't installed.