Install Scipy with MKL through PIP

Question

I am using PIP to install Scipy with MKL to accelerate the performance. My OS is Ubuntu 64 bit. Using the solution from this question, I create a file .numpy-site.cfg

[mkl]
library_dirs=/opt/intel/composer_xe_2013_sp1/mkl/lib/intel64/
include_dirs=/opt/intel/mkl/include/
mkl_libs=mkl_intel_lp64,mkl_intel_thread,mkl_core,mkl_rt
lapack_libs=

This file helps me to install Numpy with MKL successfully. However, using the same above file, installing Scipy prompts the error

ImportError: libmkl_rt.so: cannot open shared object file: No such file or directory

I also use

export LD_LIBRARY_PATH=/opt/intel/composer_xe_2013_sp1/mkl/lib/intel64

but the problem is still the same.

Anyone know how to fix this problem? I don't want to install Scipy manually so anyone give me some hints to fix it.

Have you tried with `export LD_LIBRARY_PATH=/opt/intel/composer_xe_2013_sp1/mkl/lib/intel64:$LD_LIBRARY_PATH` ? — Gabriele Salvatori, Feb 24 '14 at 15:19
I have tried this option before. However, the error is still in. I dont know how to fix it. — tndoan, Feb 25 '14 at 02:27
You have to check whether these libraries actually exist at the specified locations with `library_dirs` and `include_dirs`. There could be many reasons why this is not the case. The most obvious ones being that you installed an older/newer Intel Studio version or you installed the compiler and the libraries to a non-default location. — Felix Crazzolara, Nov 23 '19 at 07:56

score 18 · Answer 1 · answered Jun 19 '18 at 12:34

18

Intel has been publishing wheels of packages like Numpy, Scipy and Scikit-learn to PyPI. These wheels have been built while linking against Intel MKL, and include various optimizations.

If you want Scipy built with Intel MKL:

#Remove existing Numpy and/or Scipy:
pip uninstall numpy scipy -y
#Install scipy built with Intel MKL:
pip install intel-scipy

More information available here

answered Jun 19 '18 at 12:34

t7t0t0t7t

362
3
5

3

This does not work as of March 2019 for Python 3.7. Pip does not find an "intel-scipy" on the repository. – AmphotericLewisAcid Mar 27 '19 at 21:57
1

I had python 3.6.9 and it worked. You might also need to get the llibpython. `apt install libpython3.6`. I verified my installation inside python with `numpy.show_config()` – 0-_-0 Mar 11 '20 at 16:19

score 5 · Answer 2 · answered Oct 30 '16 at 15:16

2 years have passed since this question was asked.

There are now numpy/scipy wheels for linux that use a openblas compiled for avx2, so you can get much better performance without building packages. You may need to upgrade pip to get it to install the wheel:

pip install --upgrade pip
pip install numpy scipy

If you want MKL, then you can install Anaconda or Intel Distribution for Python. They use conda instead of pip to manage packages, but they are free and distribute packages that contain all the dependences, including MKL.

There isn't a wheel for Ubuntu which the OP is using. The best choice would be Anaconda. — Aniket Chowdhury, Jul 14 '17 at 17:21

score 5 · Answer 3 · answered Jul 23 '17 at 08:01

I have Win10 64Bit with Python 3.6.2 i have installed scipy through http://www.lfd.uci.edu/~gohlke/pythonlibs/#scipy

I followed following steps :

Step 1: Uninstall if you have any previous version of numpy, pip uninstall numpy
Step 2 : Download numpy‑1.13.1+mkl‑cp36‑cp36m‑win_amd64.whl with MKL(Math Kernel Library) from below link, http://www.lfd.uci.edu/~gohlke/pythonlibs/#numpy
Step 3: copy downloaded file into another location and launch command prompt from that location.
Step 4: run this command, pip install -U numpy-1.13.1+mkl-cp36-cp36m-win_amd64.whl
Step 5: Now Download scipy library from, http://www.lfd.uci.edu/~gohlke/pythonlibs/#scipy
Step 6: Copy downloaded file into same location in which numpy is copied.
Step 7: In CMD prompt run this cmd, pip install scipy-0.19.1-cp36-cp36m-win_amd64.whl

Done!

thanks a lot buddy, great answer and great representation. – Madhur Yadav Aug 02 '18 at 11:46 — Madhur Yadav, Aug 02 '18 at 11:46

score 4 · Answer 4 · answered Jul 08 '17 at 00:11

4

If you are having trouble installing or running with specific version then first uninstall and then install

Step 1:

pip uninstall -v numpy

Step 2: download the wheel file and install

pip install -U numpy-1.13.0+mkl-cp36-cp36m-win_amd64.whl

In this example wheel file name is "numpy-1.13.0+mkl-cp36-cp36m-win_amd64.whl "

answered Jul 08 '17 at 00:11

Amir Md Amiruzzaman

1,911
25
24

Please let me know what did you do? Did you follow the steps i suggested? – Amir Md Amiruzzaman Jul 12 '17 at 14:52
Sorry. It is such a long time. I have not used this server anymore so cannot apply your solution. – tndoan Jul 21 '17 at 02:34
1

OP explicitly said Ubuntu. – Zuoanqh Nov 30 '17 at 10:21

score 2 · Answer 5 · answered Oct 31 '16 at 23:52

I have been facing this problem the past few weeks on: Windows 10 64 bit Python 3.5.2

My workaround:

First: pip install wheel

Next: Download Numpy and Scipy form Gholke's repo Numpy and SciPy

Then:

pip install numpy_package.whl

pip install scipy_package.whl

score 1 · Answer 6 · answered Oct 30 '16 at 17:39

Since the actual question itself was not answered, let me give it a shot...

I think the problem here basically is that the BLAS/LAPACK libraries being used are spread out across multiple location, and numpy doesn't handle this well.

We have fixed this in EasyBuild, where we have been building numpy/scipy on top of Intel MKL for a while now, with this patch: https://github.com/hpcugent/easybuild-easyconfigs/blob/master/easybuild/easyconfigs/n/numpy/numpy-1.8.1-mkl.patch

score 0 · Answer 7 · answered Jun 08 '18 at 01:24

0

For me, MKL came with Scipy library by conda install scipy

@rscohn2 solution gave me a hint for using packages.

For the reference, my environment is included Ubuntu, Anaconda, Python 3.6, Scipy 1.1 and MKL 2018.0.

answered Jun 08 '18 at 01:24

Cloud Cho

1,594
19
22

Install Scipy with MKL through PIP

7 Answers7

Linked