14

As the final step in building a custom python, I need to add a myproject.pth.

Currently I'm doing this in a Makefile:

install:
        rm -f            ../../lib/python2.6/site-packages/myproject.pth
        cp myproject.pth ../../lib/python2.6/site-packages/myproject.pth

but I would like to encapsulate this in a setup.py. Unfortuntately the setup.py docs don't seem to cover this trivial case! Any help appreciated. I tried this but it doesn't work:

from setuptools import setup
setup(
    packages=['mypackage_pth'],
    package_dir={'mypackage_pth': '.'},
    package_data={'mypackage_pth': ['mypackage.pth']},
)
DC Slagel
  • 528
  • 1
  • 7
  • 13
Mark Harrison
  • 297,451
  • 125
  • 333
  • 465

3 Answers3

10

The right thing to do here, is to extend setuptools' build_py, and copy the pth file from the source directory into the build directory, in the location where setuptools prepares all the files that go into site-packages there.

from setuptools.command.build_py import build_py


class build_py_with_pth_file(build_py):
     """Include the .pth file for this project, in the generated wheel."""

     def run(self):
         super().run()

         destination_in_wheel = "mypackage.pth"
         location_in_source_tree = "src/mypackage.pth"
 
         outfile = os.path.join(self.build_lib, destination_in_wheel)
         self.copy_file(location_in_source_tree, outfile, preserve_mode=0)

setup(
   ...,
   cmdclass={"build_py": build_py_with_pth_file},
)

All the other answers here (at the time of writing) are wrong in subtle ways.

data_files=[(site_packages_path, ["mypackage.pth"])]

This is semantically wrong -- the pth file is NOT data. It's code, much like how the various .py files in the rest of your project are code. More importantly, this is also functionally broken -- in a slightly subtle but important manner.

This embeds the site_packages_path into the wheel. You'll end up with a wheel containing a file path like:

my_package-1.0.0.data/data/lib/python3.9/site-packages/mypackage.pth

This wheel will only work on Python 3.9 (because that's what the path is) but it will be very easy to tag as py3 (i.e. compatible with all Python versions).

This is non-trivial to detect, since you'll need a development workflow that runs tests across multiple different Python versions, using the generated wheel.

shutil.copy('mypackage.pth', site_packages_path)

This is... bad.

While it will work across Python versions, this will "install" the pth file even if a user downloads the project, with pip download mypackage.

More importantly, a wheel generated for this project will not have any pth file associated with the project. So, subsequent installations will be not get the pth file installed (since pip will cache locally built wheels).

This can't be reproduced by installing from the source directory locally and is similarly non-trivial: This is non-trivial to detect, since you'll need a development workflow that installs the package in a different environment than the one you've built it from, to run the tests in a manner that they can detect this.

pradyunsg
  • 18,287
  • 11
  • 43
  • 96
  • 1
    The proposed solution appears to work well. Another caveat for doing things the wrong way is that it breaks in certain distributions when installing as root (Ubuntu has the code end up in /usr/local/lib/python3.x/dist-packages, but the .pth ends up in /usr/local/lib/python3.x/site-packages, which isn't in sys.path). – Itamar Turner-Trauring Oct 19 '22 at 17:08
4

You're looking for the data_files parameter to setup:

from distutils import sysconfig
site_packages_path = sysconfig.get_python_lib()

setup(...,
  data_files=[(site_packages_path, ["mypackage.pth"])]
 )
stderr
  • 8,567
  • 1
  • 34
  • 50
  • The key I was missing was the `get_python_lib()`. Great tip! – bukzor Nov 06 '13 at 03:19
  • This didn't work for me. The path configuration files (`*.pth`) are copied into a package itself, not `site-packages`. Do you know what might go wrong? – astronaut Jan 09 '14 at 14:13
  • What is `site_packages_path` in your case, @astronaut? – stderr Jan 09 '14 at 17:18
  • Hi, @MikeSteder. I get it from `sysconfig.get_python_lib()`, just like you've shown above. In my case it's `d:/venv2/Lib/site-packages/`. I might have said it wrong, but by "package itself" I meant the egg folder containing all package content (not python package). – astronaut Jan 10 '14 at 10:09
  • 2
    @MikeSteder, The files end up in `d:/venv2/Lib/site-packages/foobar-2.0.SNAPSHOT-py2.5.egg/` instead of just `d:/venv2/Lib/site-packages/`. – astronaut Jan 10 '14 at 10:16
  • Is there any solution? I have same problem @astronaut has, `foobar.pth` gets into egg folder instead of `site-packages`, and it seems to be special case handling for site-packages only. – toriningen Dec 13 '15 at 10:20
  • I _really_ wanted this to work, but... it doesn't. And everything I've read indicates that it's not 'supposed' to. Spent almost an hour trying to force the issue, but to no avail. The [`site.py` docs](https://docs.python.org/2/library/site.html) state pretty clearly where Python looks for `.pth` files, and it would be pretty surprising if random packages were able to override this behavior. – evadeflow Apr 01 '16 at 19:25
0

Adapted from stderr's answer. Not sure if you're supposed to do this, but it works.

from distutils import sysconfig
site_packages_path = sysconfig.get_python_lib()

import shutil
shutil.copy('mypackage.pth', site_packages_path)

setup(...)
S. Wolf
  • 33
  • 1