In setup.py or pip requirements file, how to control order of installing package dependencies?

Question

I've got a Python package with its setup.py having dependencies declared via the usual way, in install_requires=[...]. One of the packages there, scikits.timeseries, has a setup.py expecting numpy to already be installed, thus, I'd like some way to have numpy installed first. For this case and in general, can the order of dependency installation be controlled? How?

Currently the order in which setup.py pulls down dependencies (as listed in the arg install_requires) seems practically random. Also, in the setup.py setup(...) I tried using the arg:

extras_require={'scikits.timeseries': ['numpy']}

...without success, the order of installing dependencies was unaffected.

I also tried setting up a pip requirements file, but there too, pip's order of installing dependencies didn't match the line-order of the requirements file, so no luck.

Another possibility would be to have a system call near the top of setup.py, to install numpy before the setup(...) call, but I hope there's a better way. Thanks in advance for any help.

For whomever comes by this problem. [This](https://github.com/scikit-learn/scikit-learn/issues/2569) is the reason why numpy is not included as a dependency. — Stephan Dollberg, Dec 03 '14 at 17:12

score 6 · Accepted Answer · answered Feb 15 '11 at 15:17

6

If scikits.timeseries needs numpy, then it should declare it as a dependency. If it did, then pip would handle things for you (I'm pretty sure setuptools would, too, but I haven't used it in a long while). If you control scikits.timeseries, then you should fix it's dependency declarations.

answered Feb 15 '11 at 15:17

Hank Gay

70,339
36
160
222

That makes sense, thank you. I don't control that code base, but I'll fork it. :) – limist Feb 18 '11 at 04:25
it would be nice to mention in your answer why scikit doesn't have it as an explicit dependence: https://github.com/scikit-learn/scikit-learn/issues/2569 – Charlie Parker Jan 10 '17 at 23:26
1

@CharlieParker When I answered the question, that discussion didn't exist. – Hank Gay Jan 11 '17 at 16:46
This doesn't seem to be true. I have a library being installed, where both the parent and child declare the same dependency, and the order is still random! The child lib fails to install because it's requirement is missing. The setup function works/fails randomly across environments. – BuvinJ Jan 19 '19 at 01:44

Andriy Ivaneyko · Answer 2 · 2020-07-18T07:30:52.750

4

Use setup_requires parameter, for instance to install numpy prior scipy put it into setup_requires and add __builtins__.__NUMPY_SETUP__ = False hook to get numpy installed correctly:

setup(
    name='test',
    version='0.1',
    setup_requires=['numpy'],
    install_requires=['scipy']
)

def run(self):
    __builtins__.__NUMPY_SETUP__ = False
    import numpy

edited Jul 18 '20 at 07:30

answered Jul 15 '16 at 16:02

Andriy Ivaneyko

20,639
6
60
82

1

This doesn't work. It looks like it should with a passing glance to the source code, but it doesn't. Reading the docs, it says this will download it first, but not install it first. – BuvinJ Jan 19 '19 at 01:42

score 1 · Answer 3 · answered Jan 19 '19 at 16:53

Here's a solution which actually works. It's not an overly "pleasant" method to have to resort to, but "desperate times...".

Basically, you have to:

Override the setuptools "install command" class (plus the closely related analogs)
Execute pip from the script via command line statements for which you can enforce the order

The drawbacks to this are:

Pip must be installed. You can't just execute setup.py in an environment without that.
The console output of the initial "prerequisite" installs don't appear for some weird reason. (Perhaps I'll post an update here down the line fixing that...)

The code:

from setuptools import setup

# Override standard setuptools commands. 
# Enforce the order of dependency installation.
#-------------------------------------------------
PREREQS = [ "ORDERED-INSTALL-PACKAGE" ]

from setuptools.command.install import install
from setuptools.command.develop import develop
from setuptools.command.egg_info import egg_info

def requires( packages ): 
    from os import system
    from sys import executable as PYTHON_PATH
    from pkg_resources import require
    require( "pip" )
    CMD_TMPLT = '"' + PYTHON_PATH + '" -m pip install %s'
    for pkg in packages: system( CMD_TMPLT % (pkg,) )       

class OrderedInstall( install ):
    def run( self ):
        requires( PREREQS )
        install.run( self )        

class OrderedDevelop( develop ):
    def run( self ):
        requires( PREREQS )
        develop.run( self )        

class OrderedEggInfo( egg_info ):
    def run( self ):
        requires( PREREQS )
        egg_info.run( self )        

CMD_CLASSES = { 
     "install" : OrderedInstall
   , "develop" : OrderedDevelop
   , "egg_info": OrderedEggInfo 
}        
#-------------------------------------------------

setup ( 
     ...
    install_requires = [ "UNORDERED-INSTALL-PACKAGE" ],
    cmdclass = CMD_CLASSES
)

score 0 · Answer 4 · answered Jun 20 '16 at 09:35

0

You can add numpy to setup_requires section:

setup_requires=['numpy'],

answered Jun 20 '16 at 09:35

yurez

2,826
1
28
22

I do have it there but it seems its still an issue since when it imports numpy it seems to freak out or it doesn't install it before it installs scipy. I have the requires as follow: `install_requires=['numpy','namespaces','pdb','scikit-learn','scipy']` – Charlie Parker Jan 10 '17 at 23:24

In setup.py or pip requirements file, how to control order of installing package dependencies?

4 Answers4

Linked