6

What would be a cross-platform way of shipping data_files with setup.py (compatible with pip)?

From the official documentation, one needs to write:

setup(...,
    data_files=[('bitmaps', ['bm/b1.gif', 'bm/b2.gif']),
                ('config', ['cfg/data.cfg']),
                ('/etc/init.d', ['init-script'])]
    )

and 'bitmaps', etc. are the sub-directories where those files should go (relative to sys.prefix).

However, this is sub-optimal for cross-platform installations where the standard sub-dir will depend on the system. Additionally, installing the package in developer mode will not place the files where they will later be after the installation, making this process of finding/using resources ultimately hard / annoying to debug.

I have looked into appdirs, but it seems difficult to make it work properly during installation, e.g. if using the user directory for data this gets actually tied to my development environment.

The reason I am asking this is because I have a small Python package that implements a simple GUI and I would like to ship an icon with it.

For the record, I am OK with processing setup.py with setuptools.

norok2
  • 25,683
  • 4
  • 73
  • 99
  • 1
    Note that absolute target paths won't work with `pip` anyway because all absolute `data_files` targets will be relativized to `site-packages`, see my answer [here](https://stackoverflow.com/a/47465374/2650249) for more details. For bundling resource files, I'd rather use `package_data` and place the files under some package dir. You can then reference the files with `pkg_resources.resource_file()` etc. in a cross-platform manner. – hoefling Jul 19 '18 at 10:02
  • Thanks for the hint. I have eventually used `package_data` in conjunction with `pkg_resources.resource_filename()`. Weirdly enough, using `package_data`, I am unable to build the package using Python 2 (although the resulting package is Python 2 compatible). If you make up for your comment as an answer, I'd be happy to accept it. – norok2 Jul 19 '18 at 12:31
  • Are you building a source dist or a wheel? What error do you get? If you update your question with error details (or ask a new one and link it here), I will try to help. – hoefling Jul 19 '18 at 15:44

2 Answers2

4

As suggested in the comments, for bundling resource files, I'd rather use package_data and place the files under some package dir. Example:

project
├── pkg1
│   ├── __init__.py
│   └── icons
│       └── image.png
└── pkg2
    └── __init__.py

Packaging in the setup.py script:

from setuptools import setup


setup(
    ...
    package_data={'pkg1': ['icons/image.png']},
)

Update for accessing resource files in code:

Python 3.7 introduced importlib.resources which replaces the old pkg_resources functionality and offers a modern resource machinery that utilizes pathlib:

filepath = importlib_resources.path('pkg1', 'icons/image.png')

For Python 3.6 and older, there's a backport named importlib_resources. The version-agnostic example is thus:

import sys

if sys.version_info >= (3, 7):
    from importlib import resources as importlib_resources
else:
    import importlib_resources

filepath = importlib_resources.path('pkg1', 'icons/image.png')

Use importlib_resources instead of pkg_resources where possible.

Original answer, for the history only

To reference the resource files in code, use pkg_resources:

import pkg_resources

filepath = pkg_resources.resource_filename('pkg1', 'icons/image.png')

The cross-platform support is thus handled by pkg_resources. Refer to ResourceManager API for available resource access functions.

hoefling
  • 59,418
  • 12
  • 147
  • 194
  • 1
    `sys.version` should probably by `sys.version_info` ([doc](https://docs.python.org/3/library/sys.html#sys.version_info)) – kontur Mar 17 '20 at 13:17
-1

Update Check out the solution from @hoefling

The data_files options specify the files to upload for your package.

From the docs:

No directory information from files is used to determine the final location of the installed file; only the name of the file is used.

If you want to build cross platform directory paths, use the os module

import os

path = os.path.join('root', 'directory')

If you want to determine the platform when someone tries to build your package, use the sys module

import sys

current_platform = sys.platform.lower()
is_windows = current_platform.startswith('win')
is_ubuntu =  'ubuntu' in current_platform
is_mac = 'darwin' in current_platform


if is_windows:
     ... # windows specific dayta

elif is_ubuntu:
     ....

else:
    raise Exception('Platform not supported')
Tobey
  • 1,400
  • 1
  • 10
  • 25
  • 1
    Wouldn't this define the directory at build-time (exactly like using `appdirs` would) hence rendering this is approach hardly of any help for cross-platform scenarions? – norok2 Jul 19 '18 at 09:55