5

It is my understanding that NumPy dropped support for using the Accelerate BLAS and LAPACK at version 1.20.0. According to the release notes for NumPy 1.21.1, these bugs have been resolved and building NumPy from source using the Accelerate framework on MacOS >= 11.3 is now possible again: https://numpy.org/doc/stable/release/1.21.0-notes.html, but I cannot find any documentation on how to do so. This seems like it would be an interesting thing to try and do because the Accelerate framework is supposed to be highly-optimized for M-series processors. I imagine the process is something like this:

  1. Download numpy source code folder and navigate to this folder.
  2. Make a site.cfg file that looks something like:
[DEFAULT]
library_dirs = /some/directory/
include_dirs = /some/other/directory/

[accelerate]
libraries = Accelerate, vecLib
  1. Run python setup.py build

The problem is I do not know 1. what the variables library_dirs and include_dirs should be so that NumPy knows to use Accelerate BLAS and LAPACK and 2. if there are any other additional steps that need to be taken. If anyone knows how to do this or can provide any insight, it would be greatly appreciated.

Joel
  • 53
  • 1
  • 4

2 Answers2

5

No it doesn't have to be that complicated. I used these two commands and was able to install numpy with Apple Accelerate on Mac M1.

pip install cython pybind11
pip install --no-binary :all: --no-use-pep517 numpy

Reference: How to install SciPy on Apple Silicon (ARM / M1)

Dat
  • 5,405
  • 2
  • 31
  • 32
  • 1
    I've spent my whole morning trying to setup performant numpy on an M1 Mac, and this is by far the cleanest solution I've found! I can confirm that this works under OS X 12.3, in a clean Python 3.10 virtualenv, for numpy 1.22. Using https://gist.github.com/markus-beuckelmann/8bc25531b11158431a5b09a45abd6276 as a benchmark, I was seeing total run times in the range of 80-110 seconds with various pre-built numpy wheels. A convoluted setup with conda clocked 18 seconds, which I'm treating as "best possible". @Dat's answer clocked 26 seconds, which while slower is also much simpler! – knite May 09 '22 at 16:06
  • 1
    Also, --no-use-pep517 can be omitted, I installed successfully with just the no-binary flag. – knite May 09 '22 at 16:14
  • I like this so much. Thank you! – seongjoo Mar 09 '23 at 20:36
2

I actually attempted this earlier today and these are the steps I used:

  • In the site.cfg file, put
[accelerate]
libraries = Accelerate, vecLib

  • Build with NPY_LAPACK_ORDER=accelerate python3 setup.py build

  • Install with pip3 install .

Afterwards, np.show_config() returned the following

blas_mkl_info:
  NOT AVAILABLE
blis_info:
  NOT AVAILABLE
openblas_info:
  NOT AVAILABLE
accelerate_info:
    extra_compile_args = ['-I/System/Library/Frameworks/vecLib.framework/Headers']
    extra_link_args = ['-Wl,-framework', '-Wl,Accelerate']
    define_macros = [('NO_ATLAS_INFO', 3), ('HAVE_CBLAS', None)]
blas_opt_info:
    extra_compile_args = ['-I/System/Library/Frameworks/vecLib.framework/Headers']
    extra_link_args = ['-Wl,-framework', '-Wl,Accelerate']
    define_macros = [('NO_ATLAS_INFO', 3), ('HAVE_CBLAS', None)]
lapack_mkl_info:
  NOT AVAILABLE
openblas_lapack_info:
  NOT AVAILABLE
openblas_clapack_info:
  NOT AVAILABLE
flame_info:
  NOT AVAILABLE
lapack_opt_info:
    extra_compile_args = ['-I/System/Library/Frameworks/vecLib.framework/Headers']
    extra_link_args = ['-Wl,-framework', '-Wl,Accelerate']
    define_macros = [('NO_ATLAS_INFO', 3), ('HAVE_CBLAS', None)]
Supported SIMD extensions in this NumPy install:
    baseline = NEON,NEON_FP16,NEON_VFPV4,ASIMD
    found = ASIMDHP,ASIMDDP
    not found =

and my quick test suggest significant performance boost relative to OpenBlas.

haginile
  • 456
  • 5
  • 14
  • Thanks! That worked! And it is indeed much MUCH faster. Out of curiosity, I tried doing something analogous for Scipy, but without luck. – Joel Nov 07 '21 at 19:22
  • Just tried to do this myself, but somehow the build keeps erroring out. I _think_ it has to do with `-march=native` not being supported on M1s, but I certainly don't understand how the build system works. Would you mind sharing a bit more about your setups? I basically tried `git clone numpy && cd numpy && && NPY_LAPACK_ORDER=accelerate python3 setup.py build && pip3 install .` on a new machine – user2267896 Nov 09 '21 at 20:51
  • In my opinion, the most straightforward way to get an M1-native environment setup is to get the Macosx-arm64 version of miniconda3 at: https://github.com/conda-forge/miniforge#download. Anything installed in this environment will default to building for arm64 or installing published arm64 binaries. If this is done and the necessary compilers are installed, most of what you have said sounds correct. Except I'm not sure you can create the site.cfg file in the command line like that. you have to manually make it and type in the lines above. Perhaps if you showed what error you get we can help. – Joel Nov 09 '21 at 22:25
  • You might also have to delete the `site.cfg.example` file, but that's just me guessing. It's difficult to say what is going wrong without more information about the error. Maybe you don't have one or more of the build dependencies installed. – Joel Nov 09 '21 at 22:27