How to *install* extra package data using setuptools?

Question

This looks an awful lot like this question, but it isn't quite.

The answers to that question cover how to get extra data into a source distribution and binary distribution, but don't actually address how to get package data installed when included in a source distribution, or how to get them installed in a wheel file.

Here's an example of what I mean (python 3.7.4):

% ls
MANIFEST.in README.txt  foopackage  setup.py    venv

% cat setup.py
# -*- coding: utf-8 -*-

from setuptools import setup, find_packages

setup(
    name='foopackage',
    version='0.0.1',
    description='A Package Full of Foo',
    packages=find_packages(),
    package_data={
        'foopackage': [
            'README.txt',
        ],
    },
    include_package_data=True,
)

% cat MANIFEST.in
include README.txt

% . venv/bin/activate
(venv)
% python setup.py sdist bdist_wheel
[...]
% ls dist
foopackage-0.0.1-py3-none-any.whl foopackage-0.0.1.tar.gz
(venv)
% unzip -v dist/foopackage-0.0.1-py3-none-any.whl| grep README.txt
(venv)
% tar tvzf dist/foopackage-0.0.1.tar.gz| grep README.txt
-rw-r--r--  0 matt   staff       0  8 Nov 17:21 foopackage-0.0.1/README.txt
(venv)
% deactivate

% cd ../foo

% . py/bin/activate
(py)
% pip install ../foopackage/dist/foopackage-0.0.1.tar.gz
Processing ../foopackage/dist/foopackage-0.0.1.tar.gz
Installing collected packages: foopackage
  Found existing installation: foopackage 0.0.1
    Uninstalling foopackage-0.0.1:
      Successfully uninstalled foopackage-0.0.1
  Running setup.py install for foopackage ... done
Successfully installed foopackage-0.0.1
You are using pip version 19.0.3, however version 19.3.1 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.
(py)
% find . -name README.txt
(py)
%

The README.txt file isn't included in the wheel, and despite being included in the sdist file, it is not installed when that package is installed.

I can't seem to find the magic incantation to get setuptools to actually install this file somewhere. What am I missing?

I should probably point out that the real package I'm building has to be py27 and py34+ compatible, in case that constrains the solution in any way.

Answer 1

The problem is in

package_data={
    'foopackage': [
        'README.txt',
    ],
},

With this code you declare that your foopackage contains a file README.txt (file patterns in package_data are treated relative to the packages they belong to). But it is not! The file is in the root folder, not in the foopackage/ subfolder. This is also the reason why the file is not included into your wheel(s).

By listing the file in MANIFEST.in you forced setuptools to include the file into source dist but setuptools doesn't know what to do with the file — it expects the file to be inside foopackage/.

There are 2 ways to fix the problem. 1st, you can declare relative path to the file:

package_data={
    'foopackage': [
        '../README.txt',
    ],
},

In this case the fill be installed in foopackage/../README.txt, that is, in site-packages/ directory. It could be what you want.

Or not. In which case the right solution is to move the file inside foopackage/, fix MANIFEST.in and rebuild sdist and wheel(s).