How can I attach a file outside the package to my Python package

Question

I have a web application that displays the ABOUT.md file of the project. The project has the following file tree:

project_folder/
  main_package/
    assets/icon.png
    __init__.py
    app.py
  .gitignore # And other files
  README.md
  ABOUT.md
  setup.cfg
  setup.py

In app.py I have a webserver that renders and provides the contents of the file README.md. It has a code like this:

from main_package import __file__ as mpfile

# First parent is just the folder where __init__.py is located.
ABOUT_MD = Path(mpfile).parent.parent / 'ABOUT.md'

This works without building, but if I build the wheel and install it in other environment, it stops working.

I modified the config in setup.cfg so it includes the ABOUT.md

[options.package_data]
main_package = 
  ../ABOUT.md
  assets/*

But this copies the file to the root of site_packages, which I think is a bit dirty.

I want to keep the ABOUT.md file available in the root folder, so it remains very accessible through GitHub, but I also want to be able to build and publish my package.

Idea:

Modify the build system to copy the ABOUT.md from the root to main_package/assets/ABOUT.md while creating the wheel. Then, add an if inside app.py that loads the correct file depending on where it is. The problem is that I don't know how to make the build system copy the file to this path.

Update 2022-07-18: Why not linking?

In response to @Leander

If we use a Hard Link, that info is not propagated through the version control system (git), so It will appear to be two different files in every other computer, and we would have to sync the data somehow (with hooks or any kind of system), plus it would take double space in disk.

If we use a Symlink, the used space is reduced, but the web view of the repositories can't follow symlinks, so the ABOUT.md ends unusable (it just displays the plaintext path of the symlink).

You could create a hard link of README.md to `main_package/assets/ABOUT.md` so that locally you will be able to import from this directory. Then during packaging make sure this folder is taken into account and correctly packaged so that on other systems you also will be able to import the README.md file from this directory. — Leander, Jul 14 '22 at 17:46
@Leander I updated my question to reflect why linking is not a viable option — David Davó, Jul 18 '22 at 09:58
Does this answer your question? [How to read a (static) file from inside a Python package?](https://stackoverflow.com/questions/6028000/how-to-read-a-static-file-from-inside-a-python-package) — Carlos Horn, Jul 18 '22 at 10:02
@CarlosHorn the problem is that the file is from "outside" the package. Is the ABOUT.md file in GitHub — David Davó, Aug 12 '22 at 19:37

score 0 · Answer 1 · answered Aug 12 '22 at 19:41

Symlink

Symlinking the other way around (original in ./ABOUT.md and creating a symbolic link at main_package/assets/ABOUT.md to ../../ABOUT.md works flawlessly.

Is true that GitHub (for example) doesn't follow symlinks, but the ABOUT.md in the root folder remains readable.
Wheel packaging copies the file, not the symlink, so the application can be packaged and installed in any computer.
No need to add an If or try/catch when importing the file, just main_package/assets/ABOUT.md.

score 0 · Answer 2 · answered Aug 23 '22 at 07:12

Your problem is very similar to the versioning problem of repositories. Typically, the version is provided by the git repository as tag and potentially gets a suffix from the commit hash and a dirty flag if not yet committed. A very light weight solution to this problem is provided by https://github.com/jbweston/miniver so I would recommend you to follow the idea. It basically checks the version string in a file and if the placeholder is found, it uses git to compose the version string, otherwise the given value. This value is only set during the build process, meaning it is overwritten when executing the setup.py and therefore only appears in installed instances of the software.

In your case, just replace the version with the ABOUT.md content during the build process.

How can I attach a file outside the package to my Python package

Idea:

Update 2022-07-18: Why not linking?

2 Answers2

Symlink