1

I am trying to get scipy installed inside my virtualenv. I tried installing it via apt-get python-scipy and it downloaded and installed in like 8 seconds, no problem. But to get it installed inside my virtual env I tried to install it with pip:

pip install scipy

and it churns and churns and like 5 minutes later I get this error saying that I ran out of virtual memory? Is this thing serious? Anyone have any hints on this?

building 'scipy.sparse._sparsetools' extension
    compiling C++ sources
    C compiler: x86_64-linux-gnu-g++ -pthread -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector --param=ssp-buffer-size=4 -Wformat -Werror=format-security -D_FORTIFY_SOURCE=2 -fPIC

    compile options: '-D__STDC_FORMAT_MACROS=1 -Iscipy/sparse/sparsetools -I/home/ubuntu/.virtualenvs/litem/lib/python3.4/site-packages/numpy/core/include -I/usr/include/python3.4m -I/home/ubuntu/.virtualenvs/litem/include/python3.4m -c'
    x86_64-linux-gnu-g++: scipy/sparse/sparsetools/bsr.cxx
    In file included from /home/ubuntu/.virtualenvs/litem/lib/python3.4/site-packages/numpy/core/include/numpy/ndarraytypes.h:1781:0,
                     from /home/ubuntu/.virtualenvs/litem/lib/python3.4/site-packages/numpy/core/include/numpy/ndarrayobject.h:18,
                     from scipy/sparse/sparsetools/sparsetools.h:5,
                     from scipy/sparse/sparsetools/bsr.cxx:4:
    /home/ubuntu/.virtualenvs/litem/lib/python3.4/site-packages/numpy/core/include/numpy/npy_1_7_deprecated_api.h:15:2: warning: #warning "Using deprecated NumPy API, disable it by " "#defining NPY_NO_DEPRECATED_API NPY_1_7_API_VERSION" [-Wcpp]
     #warning "Using deprecated NumPy API, disable it by " \
      ^
    virtual memory exhausted: Cannot allocate memory
    /home/ubuntu/.virtualenvs/litem/lib/python3.4/site-packages/numpy/distutils/system_info.py:635: UserWarning: Specified path  is invalid.
      warnings.warn('Specified path %s is invalid.' % d)
    Running from scipy source directory.
    /home/ubuntu/.virtualenvs/litem/lib/python3.4/site-packages/numpy/distutils/system_info.py:635: UserWarning: Specified path /home/ubuntu/.virtualenvs/litem/include/python3.4m is invalid.
      warnings.warn('Specified path %s is invalid.' % d)
    /home/ubuntu/.virtualenvs/litem/lib/python3.4/site-packages/numpy/distutils/system_info.py:635: UserWarning: Specified path /usr/local/include/python3.4m is invalid.
      warnings.warn('Specified path %s is invalid.' % d)
    error: Command "x86_64-linux-gnu-g++ -pthread -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector --param=ssp-buffer-size=4 -Wformat -Werror=format-security -D_FORTIFY_SOURCE=2 -fPIC -D__STDC_FORMAT_MACROS=1 -Iscipy/sparse/sparsetools -I/home/ubuntu/.virtualenvs/litem/lib/python3.4/site-packages/numpy/core/include -I/usr/include/python3.4m -I/home/ubuntu/.virtualenvs/litem/include/python3.4m -c scipy/sparse/sparsetools/bsr.cxx -o build/temp.linux-x86_64-3.4/scipy/sparse/sparsetools/bsr.o" failed with exit status 1
    In file included from /home/ubuntu/.virtualenvs/litem/lib/python3.4/site-packages/numpy/core/include/numpy/ndarraytypes.h:1781:0,
                     from /home/ubuntu/.virtualenvs/litem/lib/python3.4/site-packages/numpy/core/include/numpy/ndarrayobject.h:18,
                     from scipy/sparse/sparsetools/sparsetools.h:5,
                     from scipy/sparse/sparsetools/bsr.cxx:4:
    /home/ubuntu/.virtualenvs/litem/lib/python3.4/site-packages/numpy/core/include/numpy/npy_1_7_deprecated_api.h:15:2: warning: #warning "Using deprecated NumPy API, disable it by " "#defining NPY_NO_DEPRECATED_API NPY_1_7_API_VERSION" [-Wcpp]
     #warning "Using deprecated NumPy API, disable it by " \
      ^
    virtual memory exhausted: Cannot allocate memory

    ----------------------------------------
Command "/home/ubuntu/.virtualenvs/litem/bin/python3 -c "import setuptools, tokenize;__file__='/tmp/pip-build-43b9l311/scipy/setup.py';exec(compile(getattr(tokenize, 'open', open)(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" install --record /tmp/pip-7bbgu_yj-record/install-record.txt --single-version-externally-managed --compile --install-headers /home/ubuntu/.virtualenvs/litem/include/site/python3.4/scipy" failed with error code 1 in /tmp/pip-build-43b9l311/scipy
Chase Roberts
  • 9,082
  • 13
  • 73
  • 131
  • 1
    I too have run into this issue recently. My solution was to download the prebuilt packages of scipy from my distro. Archlinux had builds of scipy and numpy available already, I'd suspect that ubuntu has something similar that you could just install. This isn't a solution but a viable workaround at least since you're just running into issues building it. – Hamel Kothari Mar 07 '16 at 20:39
  • @Hamel Presumably `apt-get` is installing the pre-built package, which is why it installed so quickly. The problem is that the OP is trying to get scipy inside of a virtualenv, so pip is trying to compile the whole thing (which is huge). Did you use those prebuilt packages in a virtualenv? If so, how? – Mike Mar 07 '16 at 20:43
  • As a workaround, I highly recommend using [conda envs](http://conda.pydata.org/docs/using/envs.html) instead of virtualenvs. It's so much faster, easier, and better in my experience (at least for scientific computing, which is all I use python for). – Mike Mar 07 '16 at 20:44
  • Currently, I am trying to get this working in conjunction with my django site running on my AWS instance inside a virtualenv. I am not the greatest sysadmin, so I don't really want to try stripping out all the django stuff and redoing it inside a conda envs and reconnecting uwsgi etc... So for now, I think dealing with virtualenvs is my best option (assuming i can get scipy working). – Chase Roberts Mar 07 '16 at 20:51
  • Conda's probably a *lot* easier than you think, but I'll trust your judgment as to what's easy in your case. Anyway, the point is that pip can't install a pre-built version, so it has to compile scipy -- which is huge and takes a while in the best of circumstances. And you're getting a compiler error (not properly a `pip` or `scipy` problem). Google suggests AWS EC2 micro comes with ~600MB of memory and no virtual memory by default. That sounds like it would be a tight squeeze; try increasing it. – Mike Mar 07 '16 at 21:05
  • Oh. Take a look at [this](http://stackoverflow.com/a/20890162/1194883). – Mike Mar 07 '16 at 21:06
  • That looks very promising. I will look into Conda's as well when I have a little more time. Thanks. – Chase Roberts Mar 07 '16 at 21:17

0 Answers0