10

After installation, I would like to make soft-links to some of the configuration & data files created by installation.

How can I determine the location of a new package's files installed from within the package's setup.py?

I initially hard-coded the path "/usr/local/lib/python2.7/dist-packages", but that broke when I tried using a virtual environment. (Created by virtualenv.)

I tried distutils.sysconfig.get_python_lib(), and that works inside the virtualenv. When installed on the real system, however, it returns "/usr/lib/python2.7/dist-packages" (Note the "local" directory isn't present.)

I've also tried site.getsitepackages():

Running a Python shell from the base environment:

import site

site.getusersitepackages()

'/home/sarah/.local/lib/python2.7/site-packages'

site.getsitepackages()

['/usr/local/lib/python2.7/dist-packages', '/usr/lib/python2.7/dist-packages']

Running a Python shell from a virtual environment "testenv":

import site

site.getsitepackages()

Traceback (most recent call last):

File "", line 1, in

AttributeError: 'module' object has no attribute 'getsitepackages'

I'm running "Python 2.7.3 (default, Aug 1 2012, 05:14:39)" with "[GCC 4.6.3] on linux2" on Ubuntu. I can probably cobble something together with try-except blocks, but it seems like there should be some variable set / returned by distutils / setuptools. (I'm agnostic about which branch to use, as long as it works.)

Thanks.

Sarah Messer
  • 3,592
  • 1
  • 26
  • 43
  • one way would be to `import ` and strip the package name from `.__file__`. I'm sure there are better ways though. – ev-br Jun 10 '13 at 18:40
  • I'm likely to try that as a workaround, but I'm hoping for something that doesn't feel like a kludge. On a related note, setuptools.setup() returns an object, but it doesn't seem to have attributes describing what it did. The build/, dist/, and *.egg-info/ directories are additional places that seem appropriate for this info... but they don't have it. – Sarah Messer Jun 10 '13 at 22:35
  • Poking around in the source code for distutils found some interesting items: distutils.command.install.INSTALL_SCHEMES distutils.command.install.install.user_options distutils.command.install.install.sub_commands look like they have relevant info / functions, but I don't know how to leverage them yet... especially from within the setup.py – Sarah Messer Jun 25 '13 at 23:07

2 Answers2

1

I haven't found the "correct" way of doing this, but I have found a couple tricks that seem almost-correct. One method only works on install; the other only works if the package is already installed.

For install, I use the object returned by setuptools.setup():

from setuptools import setup
s = setup([...])
installation_path = s.command_obj['install'].install_lib

(This only works during install since you need a valid Distribution object for those attributes to exist. AFAIK, the only way to get such an object is to run setup().)

On uninstall, I use the file attribute of the package, as suggested by @Zhenya above. The only catch is that when I run ./setup.py uninstall to get rid of package, I usually have directories ./package/, ./build, ./dist, and ./package.egg-info/. (The "uninstall" option is caught by my code without calling setup(). It runs a manually-created script to delete the package files.) These can redirect the python interpreter to some place other than the globally-accessible repository I'm trying to get rid of. Here's my hack to handle that:

import imp
import sys
from subprocess import Popen
from os import getcwd
Popen('rm -r build dist *.egg-info', shell=True).wait()
oldpath = sys.path
rundir = getcwd()
sys.path.remove(rundir)
mod = imp.find_module(PACKAGE)
p = imp.load_module(PACKAGE, mod[0], mod[1], mod[2])
sys.path = oldpath
installation_path = p.__file__

(This doesn't work during install since - I think - Python only inventories modules when it starts, so find_module() won't find the just-installed package unless you exit python and come back in.)

I've tested both install and uninstall on a bare environment and a virtual environment (from virtualenv 1.9.1). I'm running Ubuntu 12.04 LTS, Python 2.7.3, setuptools 0.6c11 (in the bare environment) and setuptools 0.7.4 (in virtualenv).

Sarah Messer
  • 3,592
  • 1
  • 26
  • 43
  • I guess this doesn't work anymore. In Python 3 I get Traceback (most recent call last): File "", line 1, in File "/tmp/pip-req-build-a1c6n4hb/setup.py", line 14, in s = setup([...]) TypeError: setup() takes 0 positional arguments but 1 was given – Ben Farmer Oct 31 '18 at 13:18
  • This might not work in Python 3, but the error message you're getting indicates you've omitted required inputs for the setup(). (I left them out of my answer because they're environment-specific.) See details for Python 3 setup() at https://setuptools.readthedocs.io/en/latest/setuptools.html#configuring-setup-using-setup-cfg-files – Sarah Messer Oct 31 '18 at 14:33
  • Ah yes sorry, I was being dumb. However, I have two problems: 1. I need to know the installation path *before* running setup(), because I have to tell some extra build code within setup about this path, so it can set things like rpaths and copy files to the right place. 2. I tried to circumvent this by creating a "dummy" setup() object before my real one, that does nothing, however I anyway get the error: KeyError: 'install'. So maybe it doesn't work in Python 3 anyway. – Ben Farmer Nov 20 '18 at 14:01
  • For that, I would try manually digging through the code (and documentation) for setup(), looking for how the system determines the appropriate path... In my experience, the path appears to be determined based on system parameters, details of setup.py's invocation (like user permissions), and on Python version, and has little (nothing?) to do with the package being installed. Since you're in control of the script calling setup.py, you might be able to just experiment with a test-install and assume it will always be the same path. – Sarah Messer Feb 01 '19 at 16:03
0

This will probably not answer your question, but if you need to access the source code of a package you have installed, or any other file within this package, the best way to do it is to install this package in develop mode (by downloading the sources, putting it wherever you want and then running python setup.py develop in the base directory of the package sources). This way you know where the package is found.

rparent
  • 630
  • 7
  • 14
  • That's an interesting feature I hadn't seen before (Thanks!) but it doesn't seem to help me build a link _elsewhere_ to the code. – Sarah Messer Jun 25 '13 at 23:12