Installing scikit-learn and scipy on elastic beanstalk with pip and requirements.txt

Question

When deploying to elastic beanstalk I keep getting the error:

Partial import of sklearn during the build process.
  Traceback (most recent call last):
    File "<string>", line 1, in <module>
    File "/tmp/pip-build-Le610u/scikit-learn/setup.py", line 247, in <module>
      setup_package()
    File "/tmp/pip-build-Le610u/scikit-learn/setup.py", line 237, in setup_package
      .format(scipy_req_str, instructions))
  ImportError: Scientific Python (SciPy) is not installed.
  scikit-learn requires SciPy >= 0.9.
  Installation instructions are available on the scikit-learn website: http://scikit-learn.org/stable/install.html

My requirements.txt has both scikit-learn and scipy in it.

Why is this occurring? I don't want to be installing packages manually, and was under the impression pip would sort out the installation order for me (it's currently trying to install scikit-learn before scipy). It might be worth noting the same requirements.txt works perfectly fine on my local computer.

What can I do to fix this?

Edit: If I change the order of the entries in requirements.txt so scipy is above scikit-learn, it works just fine. I'd still like to know why this fails the other way around though, since pip freeze > requirements.txt will override it with alphabetical ordering.

Maybe relevant for you: http://stackoverflow.com/questions/2213551/installing-scipy-with-pip — colidyre, Nov 19 '15 at 14:32
@colidyre I know how to install it manually, i want to know why it's not installing in the right order when I do it with requirements.txt — krishan711, Nov 19 '15 at 14:34

score 6 · Accepted Answer · answered Nov 27 '15 at 08:48

6

Requirements files are files containing a list of items to be installed using pip:

Logically, a Requirements file is just a list of pip install arguments placed in a file.

Therefore, the entries in your requirements.txt will be processed sequentially by pip. As it is now, pip doesn't have true dependency resolution and scipy lacks proper install_requires in its setup.py, you have to tell pip install scipy first. (see "setup.py vs requirements.txt" by Donald Stufft if you are interested)

The reason why the same requirements.txt works on your local computer might be you have an installed scipy before running the script. You can verify this by creating a clean python enviroment by virtualenv on your computer and run the script again.

answered Nov 27 '15 at 08:48

Jon

1,211
13
29

Thanks for the answer and the links Jon. Is the only way to resolve this to 'manually' fix the order of requirements.txt, or can you recommend another way I can do it? – krishan711 Nov 27 '15 at 09:47
1

@krishan711 I can't find any good solution to automatically fix the order. You can try [conda](https://github.com/conda/conda) instead of pip, which is a [cross-platform, Python-agnostic binary package manager](http://technicaldiscovery.blogspot.nl/2013/12/why-i-promote-conda.html) – Jon Nov 28 '15 at 09:11

Installing scikit-learn and scipy on elastic beanstalk with pip and requirements.txt

1 Answers1