39

I use numpy and scipy in different environments (MacOS, Ubuntu, RedHat). Usually I install numpy by using the package manager that is available (e.g., mac ports, apt, yum).

However, if you don't compile Numpy manually, how can you be sure that it uses a BLAS library? Using mac ports, ATLAS is installed as a dependency. However, I am not sure if it is really used. When I perform a simple benchmark, the numpy.dot() function requires approx. 2 times the time than a dot product that is computed using the Eigen C++ library. I am not sure if this is a reasonable result.

Dharman
  • 30,962
  • 25
  • 85
  • 135
Apoptose
  • 569
  • 1
  • 5
  • 10

3 Answers3

45

numpy.show_config() doesn't always give reliable information. For example, if I apt-get install python-numpy on Ubuntu 14.04, the output of np.show_config() looks like this:

blas_info:
    libraries = ['blas']
    library_dirs = ['/usr/lib']
    language = f77
lapack_info:
    libraries = ['lapack']
    library_dirs = ['/usr/lib']
    language = f77
atlas_threads_info:
  NOT AVAILABLE
blas_opt_info:
    libraries = ['blas']
    library_dirs = ['/usr/lib']
    language = f77
    define_macros = [('NO_ATLAS_INFO', 1)]
atlas_blas_threads_info:
  NOT AVAILABLE
openblas_info:
  NOT AVAILABLE
lapack_opt_info:
    libraries = ['lapack', 'blas']
    library_dirs = ['/usr/lib']
    language = f77
    define_macros = [('NO_ATLAS_INFO', 1)]
...

It looks as though numpy is using the standard CBLAS library. However, I know for a fact that numpy is using OpenBLAS, which I installed via the libopenblas-dev package.

The most definitive way to check on *nix is to use ldd to find out which shared libraries numpy links against at runtime (I don't own a Mac, but I think you can use otool -L in place of ldd).

  • For versions of numpy older than v1.10:

    ~$ ldd /<path_to_site-packages>/numpy/core/_dotblas.so

    If _dotblas.so doesn't exist, this probably means that numpy failed to detect any BLAS libraries when it was originally compiled, in which case it simply doesn't build any of the BLAS-dependent components.

  • For numpy v1.10 and newer:

    _dotblas.so has been removed, but you can check the dependencies of multiarray.so instead:

    ~$ ldd /<path_to_site-packages>/numpy/core/multiarray.so

Looking at the version of numpy I installed via apt-get:

~$ ldd /usr/lib/python2.7/dist-packages/numpy/core/_dotblas.so 
    linux-vdso.so.1 =>  (0x00007fff12db8000)
    libblas.so.3 => /usr/lib/libblas.so.3 (0x00007fce7b028000)
    libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007fce7ac60000)
    libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007fce7a958000)
    libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007fce7a738000)
    /lib64/ld-linux-x86-64.so.2 (0x00007fce7ca40000)

/usr/lib/libblas.so.3 is actually the start of a chain of symlinks. If I follow them to their ultimate target using readlink -e, I see that they point to my OpenBLAS shared library:

~$ readlink -e /usr/lib/libblas.so.3
/usr/lib/openblas-base/libblas.so.3
ali_m
  • 71,714
  • 23
  • 223
  • 298
  • I'd appreciate it, if `ldd //numpy/core/multiarray.so` was higher up in the answer. – Martin Thoma Apr 10 '19 at 12:54
  • 17
    It looks like in numpy 1.16, multiarray was reorganized. I had to run `ldd numpy/core/_multiarray_umath.cpython-36m-x86_64-linux-gnu.so` – Quantum7 Jul 09 '19 at 12:07
6

numpy.show_config() just tells that info is not available on my Debian Linux.

However /usr/lib/python3/dist-packages/scipy/lib has a subdirectory for blas which may tell you what you want. There are a couple of test programs for BLAS in subdirectory tests.

Hope this helps.

Jorgen
  • 195
  • 1
  • 10
4

You want to check numpy.show_config() to see what libraries are configured.

With the newest mkl that should show libraries = ['mkl_rt'], before that it should contain several mkl_ libraries. See also here.

Roelant
  • 4,508
  • 1
  • 32
  • 62
JoshAdel
  • 66,734
  • 27
  • 141
  • 140