15

I am trying to create a setup.py for a project that depends on SciPy. The following setup.py reproduces this:

setup(
    name='test',
    version='0.1',
    install_requires=['scipy']
)

When installing this using python setup.py develop it generates the following error:

ImportError: No module named numpy.distutils.core

However, when I install scipy using pip, it installed it from a wheel, and it works just fine.

So, my questions is, how can I create a setup.py that depends on SciPy? Why won't setuptools install dependencies from wheels? Would this work better when using Python 3 (we plan to migrate anyway, so if it works there, I'll just wait until the migration is complete).

I am using Python 2.7.8 on Mac OS X 10.10.1 with setuptools 3.6 and pip 1.5.6.

Björn Pollex
  • 75,346
  • 28
  • 201
  • 283
  • 1
    `install_requires` always bothers me; I've had to work around it at times, but I don't have a solution here. Does `install_requires=['numpy', 'scipy']` help? –  Nov 19 '14 at 16:10
  • 1
    And perhaps this [SO question & answer](http://stackoverflow.com/questions/2087148/can-i-use-pip-instead-of-easy-install-for-python-setup-py-install-dependen) help: that may let `pip` take care of the dependencies, while you otherwise get essentially the same behaviour as `python setup.py develop`. –  Nov 19 '14 at 16:11
  • It doesn't. Apparently the order in which `setuptools` install dependencies is not specified, so it tries to install SciPy first and fails. Strangely, when I run tests using [tox](http://tox.readthedocs.org) (without the most basic `tox.ini`), it installs fine. – Björn Pollex Nov 19 '14 at 16:16
  • Just read your second comment - that would also explain why it works with tox, because it installs it just like that. Great find! – Björn Pollex Nov 19 '14 at 16:17
  • @Evert: The link in your second comment essentially solves the problem, and is also more in line with packaging best practices (and official recommendations). If you would formulate that into an answer, I'd give you some rep for it (and it would mark this question as resolved). – Björn Pollex Nov 20 '14 at 13:49
  • Will do, though I also consider it an (indirectly) duplicate question. But I also agree that, since other people may stumble on this question with similar problems, a direct (accepted) answer will help. –  Nov 20 '14 at 13:57
  • What version is your scipy? There has been attempted fix with https://github.com/scipy/scipy/pull/453 – Finn Årup Nielsen Dec 15 '14 at 14:58
  • Can you use [anaconda](https://docs.continuum.io/anaconda/index) and inform your users to use it to install those dependencies? `pip` is notoriously flaky with `numpy` and `scipy` dependencies. – pylang Jul 10 '16 at 01:04
  • Also test your installation in virtualenv with updated `pip` and `setuptools`. – pylang Jul 10 '16 at 01:25

2 Answers2

2

Ultimately, this worked for me:

#!/usr/bin/env python

from setuptools import setup, Extension
from setuptools.command.build_ext import build_ext as _build_ext

#
# This cludge is necessary for horrible reasons: see comment below and
# http://stackoverflow.com/q/19919905/447288
#
class build_ext(_build_ext):
    def finalize_options(self):
        _build_ext.finalize_options(self)
        # Prevent numpy from thinking it is still in its setup process:
        __builtins__.__NUMPY_SETUP__ = False
        import numpy
        self.include_dirs.append(numpy.get_include())

setup(
    #
    # Amazingly, `pip install scipy` fails if `numpy` is not already installed.
    # Since we cannot control the order that dependencies are installed via
    # `install_requires`, use `setup_requires` to ensure that `numpy` is available
    # before `scipy` is installed.
    #
    # Unfortunately, this is *still* not sufficient: `numpy` has a guard to
    # check when it is in its setup process that we must circumvent with
    # the `cmdclass`.
    #
    setup_requires=['numpy'],
    cmdclass={'build_ext':build_ext},
    install_requires=[
        'numpy',
        'scipy',
    ],
    ...
)
Alex Reece
  • 1,906
  • 22
  • 31
2

Use setup_requires parameter to install numpy prior scipy:

setup(
    name='test',
    version='0.1',
    setup_requires=['numpy'],
    install_requires=['scipy']
)

Note: Also Fortran compiler is required to build scipy. You can install it via brew:

brew install gcc

p.s. If you have AttributeError: 'module' object has no attribute 'get_include' take a look on Why doesn't setup_requires work properly for numpy? SO question, it suppose to fix that problem.

Community
  • 1
  • 1
Andriy Ivaneyko
  • 20,639
  • 6
  • 60
  • 82