How to check BLAS/LAPACK linkage in NumPy and SciPy?

Question

I am builing my numpy/scipy environment based on blas and lapack more or less based on this walk through.

When I am done, how can I check, that my numpy/scipy functions really do use the previously built blas/lapack functionalities?

score 315 · Accepted Answer · edited Dec 16 '19 at 11:30

315

The method numpy.show_config() (or numpy.__config__.show()) outputs information about linkage gathered at build time. My output looks like this. I think it means I am using the BLAS/LAPACK that ships with Mac OS.

>>> import numpy as np
>>> np.show_config()

lapack_opt_info:
    extra_link_args = ['-Wl,-framework', '-Wl,Accelerate']
    extra_compile_args = ['-msse3']
    define_macros = [('NO_ATLAS_INFO', 3)]
blas_opt_info:
    extra_link_args = ['-Wl,-framework', '-Wl,Accelerate']
    extra_compile_args = ['-msse3', '-I/System/Library/Frameworks/vecLib.framework/Headers']
    define_macros = [('NO_ATLAS_INFO', 3)]

edited Dec 16 '19 at 11:30

Demi-Lune

1,868
2
15
26

answered Oct 13 '13 at 21:01

davost

3,484
2
13
6

4

Given its widespread usefulness, `numpy.__config__` should really be a public API. Nonetheless, **you win this round, [davost](https://stackoverflow.com/users/2877003/davost).** – Cecil Curry Feb 05 '16 at 05:51
2

So, the mere fact that `lapack_opt_info` is shown means that numpy is linked with lapack? – DanHickstein Mar 04 '16 at 23:40
55

How do you interpret the output? – Edward Newell Apr 30 '16 at 17:21
19

@CecilCurry You can use `numpy.show_config()`, which is probably a public API function by virtue of the absence of starting underscores. But it's not documented online and has no docstring, so it's no surprise that it's so hard to find. Hopefully they'll fix that. – Praveen Feb 23 '17 at 05:17
8

how do I find out which package is actually used when multiple packages are displayed? – Jonasson Mar 09 '17 at 12:53
@davost, how did you install `numpy`? Which version of Python are you using? – BallpointBen May 06 '18 at 07:10
I want just to know if my numpy is linked to blas so that linear algebra computation are fast enough. I get this: https://imgur.com/a/SsrDqg5. How do you interpret this? – seralouk Oct 11 '19 at 21:48
Be careful: this information is gathered at **build time**. As BLAS and LAPACK are standard interfaces, it's possible to build against netlib's reference implementation (which is slow) while dynamically linking to the optimized OpenBLAS implementation at runtime. – mdeff Apr 02 '20 at 20:30

score 29 · Answer 2 · edited May 19 '15 at 17:39

29

What you are searching for is this: system info

I compiled numpy/scipy with atlas and i can check this with:

import numpy.distutils.system_info as sysinfo
sysinfo.get_info('atlas')

Check the documentation for more commands.

edited May 19 '15 at 17:39

Karalga

495
4
11

answered Jan 25 '12 at 12:24

rabra

746
1
6
11

36

This doesn't seem to show whether numpy currently uses ATLAS, just whether ATLAS will be linked against during the next numpy compilation. I had numpy compiled before ATLAS. It worked very slow until I recompiled numpy (sure thing), but both before and after numpy recompilation sysinfo.get_info('atlas') shown the same output. How to check the current state of affairs? – dmytro Jan 19 '13 at 15:05
7

How to interpret the output? – Eric O. Lebigot Nov 08 '13 at 11:50
3

You might have 'blas' instead of 'atlas' installed (this happens if you install openblas on debian based distros). – Malcolm Nov 19 '13 at 19:24
4

[davost](https://stackoverflow.com/users/2877003/davost)'s answer should probably have been accepted in lieu of this response, which fails to address the question at hand. Since this response does uncover interesting machinery of use in solving _other_ related questions, a hearty thanks all the same! – Cecil Curry Feb 05 '16 at 05:44
2

@rabra `sysinfo.get_info('atlas')` returned nothing for me but `sysinfo.get_info('blas')` returned ```{'include_dirs': ['/usr/local/include', '/usr/include', '/opt/local/include', '/usr/local/Cellar/python/2.7.13/Frameworks/Python.framework/Versions/2.7/include'], 'libraries': ['blas', 'blas'], 'library_dirs': ['/usr/lib']}``` and `sysinfo.get_info('lapack')` returned ```{'language': 'f77', 'libraries': ['lapack', 'lapack'], 'library_dirs': ['/usr/lib']}``` What does it mean ? – SebMa Jun 26 '17 at 16:59
This returns nothing but an empty set – endolith Sep 04 '19 at 02:47
This doesn't work any more, it's deprecated and returns an empty dict. At import you get the helpful message "`numpy.distutils` is deprecated since NumPy 1.23.0, as a result of the deprecation of `distutils` itself. It will be removed for Python >= 3.12. For older Python versions it will remain present." – Cris Luengo Jun 30 '23 at 16:27

talonmies · Answer 3 · 2012-01-25T12:11:32.097

You can use the link loader dependency tool to look at the C level hook components of your build and see whether they have external dependencies on your blas and lapack of choice. I am not near a linux box right now, but on an OS X machine you can do this inside the site-packages directory which holds the installations:

$ otool -L numpy/core/_dotblas.so 
numpy/core/_dotblas.so:
    /System/Library/Frameworks/Accelerate.framework/Versions/A/Accelerate (compatibility version 1.0.0, current version 4.0.0)
    /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 125.2.0)
    /System/Library/Frameworks/vecLib.framework/Versions/A/vecLib (compatibility version 1.0.0, current version 268.0.1)

$ otool -L scipy/linalg/flapack.so 
scipy/linalg/flapack.so (architecture i386):
    /System/Library/Frameworks/Accelerate.framework/Versions/A/Accelerate (compatibility version 1.0.0, current version 4.0.0)
    /usr/local/lib/libgcc_s.1.dylib (compatibility version 1.0.0, current version 1.0.0)
    /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 111.1.4)
    /System/Library/Frameworks/vecLib.framework/Versions/A/vecLib (compatibility version 1.0.0, current version 242.0.0)
scipy/linalg/flapack.so (architecture ppc):
    /System/Library/Frameworks/Accelerate.framework/Versions/A/Accelerate (compatibility version 1.0.0, current version 4.0.0)
    /usr/local/lib/libgcc_s.1.dylib (compatibility version 1.0.0, current version 1.0.0)
    /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 111.1.4)

$ otool -L scipy/linalg/fblas.so 
scipy/linalg/fblas.so (architecture i386):
    /System/Library/Frameworks/Accelerate.framework/Versions/A/Accelerate (compatibility version 1.0.0, current version 4.0.0)
    /usr/local/lib/libgcc_s.1.dylib (compatibility version 1.0.0, current version 1.0.0)
    /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 111.1.4)
    /System/Library/Frameworks/vecLib.framework/Versions/A/vecLib (compatibility version 1.0.0, current version 242.0.0)
scipy/linalg/fblas.so (architecture ppc):
    /System/Library/Frameworks/Accelerate.framework/Versions/A/Accelerate (compatibility version 1.0.0, current version 4.0.0)
    /usr/local/lib/libgcc_s.1.dylib (compatibility version 1.0.0, current version 1.0.0)
    /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 111.1.4)

substitute ldd in place of otool on a gnu/Linux system and you should get the answers you need.

What if there is no file `numpy/core/_dotblas.so` ? (see comment below Ricardos answer) — Woltan, Jan 25 '12 at 12:13
@Woltan: either something is seriously broken, or you are looking in the wrong place. On every Linux and OS X numpy install I have ever seen, there will be a `_dotblas.so` which is the interface wrapper to whatever blas has been used to build the distribution. On windows it will be called `_dotblas.pyd`, but the function is the same. — talonmies, Jan 25 '12 at 12:18
It seems like `_dotblas.so` is only built if you're using a `[atlas]` section in `site.cfg` (and a CBLAS-enabled BLAS library). So, you should use that, even if you're not using ATLAS (except when you're using Intel MKL, that has a dedicated section). — Kenneth Hoste, May 10 '13 at 18:44
Indeed, when no BLAS is available when NumPy is built, it builds its own dot-product routines. These can be two orders of magnitude slower than ATLAS. — Fred Foo, Sep 01 '13 at 12:33
[`_dotblas.so` no longer exists in numpy v1.10 and newer](http://docs.scipy.org/doc/numpy-dev/release.html#dropped-support), but you can check the linkage of `multiarray.so` instead — ali_m, Oct 14 '15 at 22:11
Here's a nice way (it seems): `nm /path/to/numpy/core/multiarray.so | grep blas_dot`. If there's no `blas_dot` symbol, then numpy is not linked against BLAS (for v1.10 and up) — Kenneth Hoste, Dec 03 '15 at 12:50

M.T · Answer 4 · 2019-08-27T07:42:14.253

9

You can display BLAS, LAPACK, MKL linkage using show_config():

import numpy as np
np.show_config()

Which for me gives output:

mkl_info:
    libraries = ['mkl_rt', 'pthread']
    library_dirs = ['/my/environment/path/lib']
    define_macros = [('SCIPY_MKL_H', None), ('HAVE_CBLAS', None)]
    include_dirs = ['/my/environment/path/include']
blas_mkl_info:
    libraries = ['mkl_rt', 'pthread']
    library_dirs = ['/my/environment/path/lib']
    define_macros = [('SCIPY_MKL_H', None), ('HAVE_CBLAS', None)]
    include_dirs = ['/my/environment/path/include']
blas_opt_info:
    libraries = ['mkl_rt', 'pthread']
    library_dirs = ['/my/environment/path/lib']
    define_macros = [('SCIPY_MKL_H', None), ('HAVE_CBLAS', None)]
    include_dirs = ['/my/environment/path/include']
lapack_mkl_info:
    libraries = ['mkl_rt', 'pthread']
    library_dirs = ['/my/environment/path/lib']
    define_macros = [('SCIPY_MKL_H', None), ('HAVE_CBLAS', None)]
    include_dirs = ['/my/environment/path/include']
lapack_opt_info:
    libraries = ['mkl_rt', 'pthread']
    library_dirs = ['/my/environment/path/lib']
    define_macros = [('SCIPY_MKL_H', None), ('HAVE_CBLAS', None)]
    include_dirs = ['/my/environment/path/include']

edited Aug 27 '19 at 07:42

answered Jul 19 '18 at 10:17

M.T

4,917
4
33
52

3

How do you interpret `('HAVE_CBLAS', None)]`? – seralouk Jul 26 '19 at 20:30
4

@serafeim See [link](https://github.com/python/cpython/blob/master/Lib/distutils/extension.py#L37). It essentially means that `HAVE_CBLAS` is being defined but has no value (think C: `#define HAVE_CBLAS`). It does not need a value as it is only used as a flag. I would interpret it as `HAVE_CBLAS=True`. If you did not have CBLAS, you would not have the tuple there at all. – M.T Jul 27 '19 at 18:19
I want just to know if my numpy is linked to blas so that linear algebra computation are fast enough. I get this: https://imgur.com/a/SsrDqg5. How do you interpret this? – seralouk Oct 11 '19 at 21:48
@makis numpy is linked to openblas – M.T Oct 14 '19 at 20:56

Dan T · Answer 5 · 2020-06-15T20:15:13.663

If you installed anaconda-navigator (at www.anaconda.com/anaconda/install/ for linux, Windows or macOS) - blas, scipy and numpy will all be installed and you can see them by clicking environments tab on left side of navigator home page (look for each directory in alpha order). Installing full anaconda (as opposed to miniconda or individual packages) will take care of installing many of the essential packages needed for data science.

How to check BLAS/LAPACK linkage in NumPy and SciPy?

5 Answers5

Linked