3

I am developing a python package where I want to supply custom --install-option flags to skip some installation steps if they are provided by the user already.

To do this I followed the instructions in this thread:

How to obtain arguments passed to setup.py from pip with '--install-option'?

Now this initially seemed to work, but only because all my requirements in the requirements.txt where already present.

When I retry it from a clean installation, the --install-option gets also propagated to any install calls of the dependent packages which then fail to install.

My switch in this case is --skipRLibraries. This is the command I use to install the package:

 pip install -vvv . --global-option="--skipRLibraries"

It collects all dependencies and immediately fails when starting to install them:

 Installing collected packages: joblib, pysam, pybedtools, sortedcontainers, intervaltree, python-dateutil, pytz, numpy, pandas, biopython, Cython, slamdunk
 Running setup.py install for joblib ... error
Complete output from command /usr/bin/python -u -c "import setuptools, tokenize;__file__='/tmp/pip-build-OhpiVV/joblib/setup.py';exec(compile(getattr(tokenize, 'open', open)(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" --skipRLibraries install --record /tmp/pip-0Ca3kV-record/install-record.txt --single-version-externally-managed --compile:
usage: -c [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...]
   or: -c --help [cmd1 cmd2 ...]
   or: -c --help-commands
   or: -c cmd --help

error: option --skipRLibraries not recognized

----------------------------------------
 Command "/usr/bin/python -u -c "import setuptools, tokenize;__file__='/tmp/pip-build-OhpiVV/joblib/setup.py';exec(compile(getattr(tokenize, 'open', open)(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" --skipRLibraries install --record /tmp/pip-0Ca3kV-record/install-record.txt --single-version-externally-managed --compile" failed with error code 1 in /tmp/pip-build-OhpiVV/joblib/

This is my Install command extension:

from setuptools.command.install import install as _install

class install(_install):

user_options = _install.user_options + [
    ('externalNGM', None, None),
    ('externalSamtools', None, None),
    ('skipRLibraries', None, None)
]

def initialize_options(self):
    _install.initialize_options(self)
    self.externalNGM = None
    self.externalSamtools = None
    self.skipRLibraries = None

def finalize_options(self):
    _install.finalize_options(self)

def run(self):
    global externalNGM, externalSamtools
    externalNGM = self.externalNGM
    externalSamtools = self.externalSamtools
    skipRLibraries = self.skipRLibraries
    #from subprocess import call
    #call(["pip install -r requirements.txt --no-clean"], shell=True)
    _install.run(self)
    self.execute(_runExternalBuilds, (self.install_lib, externalNGM, externalSamtools, skipRLibraries),msg="Installing external dependencies")

And of course I added it to my setup():

    cmdclass={'install': install},

Any idea how to restrict the flag only to the installation of my actual module?

Edit:

I ran over this issue also on in the pip issues section on Github:

https://github.com/pypa/pip/issues/1883

The only resolution was to first install the package with all requirements, and then reinstall it using the install flags: pip install foo pip install --no-deps --force-reinstall --upgrade --install-option="--do-some-magic" foo

But that's a) super stupid and b) not really applicable to my module since it comes with external software in default and the switch is meant to skip the installation of this external module. So I would again install it in the first place, before removing it again.

Community
  • 1
  • 1
fh_fingolfin
  • 103
  • 5

1 Answers1

0

I know this is an older issue, but I have been been exploring around with it recently and ... it seems there is no good solution for releasing a package with a custom user_options, since this would propagate.

The "workaround" would be to pass these values through Env variables to your custom install command.

from setuptools.command.install import install as _install

class install(_install):

def run(self):
    _install.run(self)
    self.execute(
        _runExternalBuilds,
        (
            self.install_lib, 
            os.environ.get("EXTERNAL..."),
            os.environ.get("SKIP_R"),
        ),
        msg="Installing external dependencies"
    )

and set up these variables when calling the package.

EXTERNAL...=True, SKIP_R=False pip install .

This has been done in Pyspark https://github.com/apache/spark/blob/master/python/setup.py (line 128)

Gabriel Azevedo
  • 21
  • 2