24

TL;DR how to link ATLAS/MKL to existing Numpy without rebuilding.

I have used Numpy to calculate with the large matrix and I found that it is very slow because Numpy only use 1 core to do calculation. After doing a lot of search I figure that my Numpy does not link to some optimized library like ATLAS/MKL. Here is my config of numpy:

>>>import numpy as np
>>>np.__config__.show()
blas_info:
    libraries = ['blas']
    library_dirs = ['/usr/lib']
    language = f77
lapack_info:
    libraries = ['lapack']
    library_dirs = ['/usr/lib']
    language = f77
atlas_threads_info:
    NOT AVAILABLE
blas_opt_info:
    libraries = ['blas']
    library_dirs = ['/usr/lib']
    language = f77
    define_macros = [('NO_ATLAS_INFO', 1)]
atlas_blas_threads_info:
  NOT AVAILABLE
openblas_info:
  NOT AVAILABLE
lapack_opt_info:
    libraries = ['lapack', 'blas']
    library_dirs = ['/usr/lib']
    language = f77
    define_macros = [('NO_ATLAS_INFO', 1)]
atlas_info:
  NOT AVAILABLE
lapack_mkl_info:
  NOT AVAILABLE
blas_mkl_info:
  NOT AVAILABLE
atlas_blas_info:
  NOT AVAILABLE
mkl_info:
  NOT AVAILABLE

For this reason, I want to link ATLAS/MKL to Numpy. However, my Numpy is installed from PIP so I don't want to install manually because I want to use the latest version. I have done some search but they are only for building from scratch. For this reason, my question are:

  • Are there any way to link ATLAS/MKL to Numpy without rebuilding again?
  • I have found that the config info is saved in _config_.py in the installed folder of Numpy. So will modifying it solve my problem? If yes, would you please show me how?
ali_m
  • 71,714
  • 23
  • 223
  • 298
tndoan
  • 643
  • 2
  • 6
  • 12

2 Answers2

28

Assuming you're running some flavour of linux, here's one way you could do it:

  1. Find out what BLAS library numpy is currently linked against using ldd.

    • For versions of numpy older than v1.10:

      $ ldd /<path_to_site-packages>/numpy/core/_dotblas.so

      For example, if I install numpy via apt-get, it links to

      ...
libblas.so.3 => /usr/lib/libblas.so.3 (0x00007fed81de8000)
...

      If _dotblas.so doesn't exist, this probably means that numpy failed to detect any BLAS libraries when it was originally installed, in which case it simply doesn't build any of the BLAS-dependent components. This often happens if you install numpy using pip without manually specifying a BLAS library (see below). I'm afraid you'll have no option but to rebuild numpy if you want to link against an external BLAS library.

    • For numpy v1.10 and newer:

      _dotblas.so has been removed from recent versions of numpy, but you should be able to check the dependencies of multiarray.so instead:

      $ ldd /<path_to_site-packages>/numpy/core/multiarray.so

  2. Install ATLAS/MKL/OpenBLAS if you haven't already. By the way, I would definitely recommend OpenBLAS over ATLAS - take a look at this answer (although the benchmarking data is now probably a bit out of date).

  3. Use update-alternatives to create a symlink to the new BLAS library of your choice. For example, if you installed libopenblas.so into /opt/OpenBLAS/lib, you would do:

    $ sudo update-alternatives --install /usr/lib/libblas.so.3 \
                                     libblas.so.3 \
                                     /opt/OpenBLAS/lib/libopenblas.so \
                                     50

    You can have multiple symlinks configured for a single target library, allowing you to manually switch between multiple installed BLAS libraries.

    For example, when I call $ sudo update-alternatives --config libblas.so.3, I can choose between one of 3 libraries:

      Selection    Path                                    Priority   Status
------------------------------------------------------------
  0            /opt/OpenBLAS/lib/libopenblas.so         40        auto mode
  1            /opt/OpenBLAS/lib/libopenblas.so         40        manual mode
  2            /usr/lib/atlas-base/atlas/libblas.so.3   35        manual mode
* 3            /usr/lib/libblas/libblas.so.3            10        manual mode

If you really want the "newest" version of numpy, you could also take a look at my answer on compiling numpy from source with OpenBLAS integration.

Installing numpy with BLAS support using pip

As @tndoan mentioned in the comments, it's possible to make pip respect a particular configuration for numpy by placing a config file in ~/.numpy-site.cfg - see this answer for more details.

My personal preference is to configure and build numpy by hand. It's not particularly difficult, and it gives you better control over numpy's configuration.

Community
  • 1
  • 1
ali_m
  • 71,714
  • 23
  • 223
  • 298
  • thanks for your help. After going to the folder _core_ in `numpy.__path__`, I cannot find _dotblas.so. I have used PIP to install Numpy so is it in other folder? – tndoan Feb 10 '14 at 10:42
  • Your answer is the same as [what I found in Internet](http://williamjohnbert.com/2012/03/how-to-install-accelerated-blas-into-a-python-virtualenv/). I will try to build it manually. Thanks for your support. – tndoan Feb 10 '14 at 12:13
  • 2
    For people who still want to use `pip` with the config file to reinstall, [here](http://stackoverflow.com/questions/13769936/supplying-numpy-site-cfg-arguments-to-pip) is the solution for them. @ali_m: would you please edit your post to put this link in your answer. Thanks – tndoan Feb 10 '14 at 12:23
  • About the advice to use openBlas, I found this at http://www.scipy.org/scipylib/building/linux.html . As of Jan. 2014 ATLAS is the recommended library to use as OpenBLAS will deadlock when used in combination with the multiprocessing module (or any use of os.fork) and older versions (<= 0.2.8) tend to crash when used on larger problem sizes. – LGenzelis Mar 30 '16 at 19:02
  • @LGenzelis I believe that issue was fixed in [this PR](https://github.com/xianyi/OpenBLAS/pull/343), dated 19th Feb 2014 (see the discussion [here](https://github.com/xianyi/OpenBLAS/issues/294) as well). Just a couple of days ago I built the latest versions of numpy (v1.11.0) and OpenBLAS (v0.2.17) and I've been using them in code that uses `multiprocessing` without issue. Unfortunately that part of the numpy documentation has not been updated in a long time. – ali_m Mar 30 '16 at 22:51
  • @ali_m, thanks for the info. Still, I wouldn't recommend using OpenBLAS. After reading your answer, I installed it (from apt-get) and recompiled scipy and numpy. After that, my code started running 3x slower, so I went back to ATLAS. – LGenzelis Apr 01 '16 at 01:27
  • @LGenzelis There's a good reason why I recommend building OpenBLAS from source. The binaries available from package maintainers are typically much older versions (they might still contain the fork bug you mentioned), and since they're not natively compiled they may not use all of the cores and instruction sets available for your particular CPU (especially if it's a fairly new model). If you care deeply about performance then building ATLAS from source would also be better than installing it via `apt-get`, but ATLAS is *much* more of a pain to compile and in my experience slower than OpenBLAS. – ali_m Apr 01 '16 at 07:02
  • @ali_m ok, when I have the time I will try to compile both of them (ATLAS and OpenBLAS) and see how they perform. – LGenzelis Apr 01 '16 at 17:55
1

The answer depends on how NumPy was built initially. If it was built against BLAS and LAPACK, then at least there is no way to force numpy.dot to use ATLAS/MKL later without rebuilding. Other functions do not use numpy.dot and you can use update-alternatives to change the targets of the symlinks libblas.so.3 and liblapack.so.3. This is because numpy.dot requires ATLAS styled CBLAS, or OpenBLAS/MKL, but not the BLAS/CBLAS and LAPACK from netlib.

I'm using openSUSE and I've installed the standard cblas-devel from netlib. However it seems just impossible to force NumPy to use the shipped cblas/cblas-devel. That is, if you built NumPy against netlib BLAS/LAPACK/CBLAS (as the official package), then _dotblas.so(which provides the BLAS version of numpy.dot) cannot be built (before 1.10), or multiarray.so(1.10 and later) does not link to libblas.so.3 at all. See the issue on github: https://github.com/numpy/numpy/issues/1265 and the cited Debian bug report: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=464784. Maybe someone can dive into the source code to make a patch...Anyway, it's just one function that is affected (numpy.dot) and you can always rebuild the whole NumPy easily using the faster OpenBLAS now, so probably no big deal after all.

Conclusion: You can link to ATLAS/MKL/OpenBLAS later without rebuilding, but numpy.dot will still be extremely slow if NumPy was not built against ATLAS/MKL/OpenBLAS initially (because numpy.dot simply didn't use any BLAS in the first place and you can do nothing about that once the compiling was done).

Update: Actually you can force numpy to build _dotblas.so. I've made a patch for numpy-1.9.2:

diff -Npru numpy-1.9.2.orig/numpy/core/setup.py numpy-1.9.2/numpy/core/setup.py
--- numpy-1.9.2.orig/numpy/core/setup.py        2015-02-01 11:38:25.000000000 -0500
+++ numpy-1.9.2/numpy/core/setup.py     2016-03-28 01:31:12.948885383 -0400
@@ -953,8 +953,8 @@ def configuration(parent_package='',top_
     #blas_info = {}
     def get_dotblas_sources(ext, build_dir):
         if blas_info:
-            if ('NO_ATLAS_INFO', 1) in blas_info.get('define_macros', []):
-                return None # dotblas needs ATLAS, Fortran compiled blas will not be sufficient.
+            #if ('NO_ATLAS_INFO', 1) in blas_info.get('define_macros', []):
+            #    return None # dotblas needs ATLAS, Fortran compiled blas will not be sufficient.
             return ext.depends[:3]
         return None # no extension module will be built

Now that _dotblas.so is linked to libblas.so.3, you can use update-alternatives to test the difference.

Molec
  • 11
  • 2