9

I'm working on a package P with setuptools and pkg_resources, where the package, after installation, needs to download some binaries and place them in a dedicated directory (P/bin/).

I'm trying to use pkg_ressources.resource_filename to get the absolute directory path. (in order to work with virtualenv)

During the installation using python setup.py install, the pkg_ressources.resource_filename doesn't return a path like /home/user/tests/venv/lib/python3.4/site-package/P/bin/, but the path to the actual module, like /home/user/projects/P/P/bin/.

That's a problem, because i need the installation directory (inside the virtualenv), not my personal project directory (where i develop the module).

If i try to pass through pypi, using pip install module, the directory returned by pkg_ressources.resource_filename is a temporary file like /tmp/pip-build-xxxxxxx/P/bin/, which is again not the place where the binaries should be put.

Here is my setup.py:

from setuptools import setup
import os

from setuptools.command.install import install as _install
from pkg_resources import resource_filename


def post_install():
    """Get the binaries online, and give them the execution permission"""
    package_dir_bin = resource_filename('P', 'bin') # should be /home/user/tests/venv/lib/python3.4/site-package/P/bin/
    # package_dir_bin = resource_filename(Requirement.parse('P'), 'bin') # leads to same results
    put_binaries_in(package_dir_bin)
    os.system('chmod +x ' + package_dir_bin + '*')



class install(_install):
    # see http://stackoverflow.com/a/18159969

    def run(self):
        """Call superclass run method, then downloads the binaries"""
        _install.run(self)
        self.execute(post_install, args=[], msg=post_install.__doc__)


setup(
    cmdclass={'install': install},
    name = 'P',
    # many metadata
    package_dir = { 'P' : 'P'},
    package_data = {
        'P' : ['bin/*.txt'] # there is an empty txt file in bin directory
    },
)

Is there a standard way to get the installation directory during the installation, cross-platform and compatible python 2 and 3 ? If not, what should i do ?

MervS
  • 5,724
  • 3
  • 23
  • 37
aluriak
  • 5,559
  • 2
  • 26
  • 39

2 Answers2

2

A workaround is to use the site package instead of pkg_resources that seems not designed for access resources during installation.

Here is a function that detect the installation directory during installation:

import os, sys, site

def binaries_directory():
    """Return the installation directory, or None"""
    if '--user' in sys.argv:
        paths = (site.getusersitepackages(),)
    else:
        py_version = '%s.%s' % (sys.version_info[0], sys.version_info[1])
        paths = (s % (py_version) for s in (
            sys.prefix + '/lib/python%s/dist-packages/',
            sys.prefix + '/lib/python%s/site-packages/',
            sys.prefix + '/local/lib/python%s/dist-packages/',
            sys.prefix + '/local/lib/python%s/site-packages/',
            '/Library/Python/%s/site-packages/',
        ))

    for path in paths:
        if os.path.exists(path):
            return path
    print('no installation path found', file=sys.stderr)
    return None

This solution is not Python 2.7 compatible, in case of installation using virtualenv, because of the known bug about the module site. (see related SO)

Community
  • 1
  • 1
aluriak
  • 5,559
  • 2
  • 26
  • 39
  • I seem to get false positives with this. I don't know why they both exist, but I seem to have both '/usr/lib/python%s/dist-packages/' AND /usr/local/lib/python%s/dist-packages/'. I want to auto-locate the latter, however your script finds the former first, so I get the wrong path. Is there no robust way to find the path that is actually going to be used by Python itself? This function seems to kind of just guess and hope for the best. – Ben Farmer Nov 20 '18 at 12:36
  • There is a lot of definition of *workaround*, one may be the one you propose: *just guess and hope for the best*. In your case, you may want to change the order in the inner list of paths, which obviousily depends of targeted OS. – aluriak Nov 21 '18 at 14:27
  • 1
    Yeah, this is unfortunately too dodgy for my needs. I will resort to a solution where these things are found at runtime. It is quite bizarre that this information cannot be found in a standard way... – Ben Farmer Nov 21 '18 at 14:42
2

The easiest solution is to follow the first snippet of this answer.

Essentially you just have to save the setup call in a variable and then look at its attributes. There are a few convenient ones

from setuptools import setup

s = setup(
    # ...
)

print(s.command_obj['install'].__dir__())

# [..., 'install_base', 'install_lib', 'install_script', ...]

Those I showed are, respectively, the equivalents of /usr, /usr/lib/pythonX.Y/site-packages and /usr/bin. But there are also other attributes that might be useful. Those are all absolute paths.

MannyC
  • 288
  • 2
  • 11