7

Please, I do need a light here. I want to install numpy using a good BLAS/LAPACK lib on Windows, but absolutely no page explains the process well enough. It seems OpenBLAS is a good and fast option.

The goal is to use "theano" with "keras", and "theano" requires that the libraries be "dynamic", not static. (Not sure I understand what that means, but it causes slowness and memory issues)

Please treat me as a complete newbie. Give me a step by step tutorial on how to do it! Don't forget to tell me "where" files should go! Which folders should go in PATH! What commands exactly I should call, and what are their output, where? What do I do with their results or with compiled files? How does numpy find them? Etc. All the sites I've seen seem to think I'm a linux expert and already know everything.

What I have tried:

  • Downloaded the compiled version of numpy+mkl from here -- This does install numpy, it becomes usable, but theano presents the memory leak problem, besides working slowly. Is it a matter of setting the right ldflags in the .theanorc file? If so, which are the flags? - About the MKL libraries, this answer may be useful?

  • Tried installing Anaconda - it doesn't work either, and I had no idea about what went wrong. It gave me messages suggesting installing some extra stuff, it worked but incredibly slow. More than 10 times slower than my bugged numpy version mentioned above (so, unacceptable, impossible to work at that speed). If I have to go changing everything about Anaconda, it's better to use a regular python instead and know what is going on.

  • Found these already compiled BLAS/LAPACK libraries (.dll and .lib) files. But.... what am I supposed to do with them? -- Simply adding their folders to the PATH var and installing numpy gives me "numpy-atlas", not the libraries I downloaded. How do I make numpy see them?

  • Tried to understand this page, but yet, it seems it will lead me exactly to the previous case, what will I do with the results? Where are the libs they suggest I use? What are the suggested quickbuild scripts, where are they?

  • Found the Cygwin option here. I haven't tried it, but it sounds it should be something easier than reinstalling all my python and packages, all from Cygwin

Daniel Möller
  • 84,878
  • 18
  • 192
  • 214
  • Meanwhile, I've been using Tensorflow instead of theano, and its performance seems better, besides not blowing my memory up (although I've heard theano sacrifices compiling time to create models that train faster....) – Daniel Möller Aug 24 '17 at 21:37
  • There is still no proper solution to this, is there? Why is conda making it so difficult to use an open-source software as opposed to intel's proprietary MKL? – Shihab Shahriar Khan Jun 29 '20 at 13:23
  • also interesting: https://stackoverflow.com/q/28190534, https://docs.anaconda.com/mkl-optimizations/index.html#uninstalling-mkl, https://github.com/conda-forge/numpy-feedstock/issues/84 – djvg Mar 30 '21 at 13:52
  • and this: https://github.com/conda-forge/scipy-feedstock/issues/80 – djvg Mar 30 '21 at 14:09
  • and the section on [accelerated linear algebra libraries](https://numpy.org/install/#numpy-packages--accelerated-linear-algebra-libraries) in the numpy docs – djvg Apr 06 '21 at 14:08
  • @ShihabShahriarKhan and all others ending up here: there is actually a clean and working solution with anaconda on Windows 10, see my answer below. – Thomas Jun 13 '21 at 00:54
  • There's also this: [How-to-use-OpenBLAS-in-Microsoft-Visual-Studio](https://github.com/xianyi/OpenBLAS/wiki/How-to-use-OpenBLAS-in-Microsoft-Visual-Studio#1-native-msvc-abi) – djvg Nov 25 '21 at 13:05

4 Answers4

5

Year 2021: this works with miniconda on Windows 10

conda create -n openblas python=3.8
conda activate openblas
conda install conda-forge::blas=*=openblas
conda install -c conda-forge numpy

Tested also with matplotlib-base.

Thomas
  • 185
  • 1
  • 7
  • 1
    Do yo know if the improvements are still there? – Ivan Gonzalez Aug 26 '21 at 22:42
  • Great, thanks, this actually seems to work! Also see https://stackoverflow.com/q/9000164, https://stackoverflow.com/q/37184618 – djvg Nov 25 '21 at 14:21
  • related: https://stackoverflow.com/a/61220181 – djvg Nov 25 '21 at 15:03
  • Hmm... this downloads OpenBLAS libraries, but numpy still links to just the normal 'blas', 'cblas', 'lapack', 'blas', 'cblas', 'lapack', 'blas', 'cblas', 'lapack' libraries instead of OpenBLAS. I did use Anaconda but it shouldn't really make a difference. No MKL libs though. I wouldn't care except Pythran requires OpenBLAS apparently. – Matt Feb 15 '22 at 22:53
3

Tried following the building instructions in http://scipy.github.io/devdocs/building/windows.html a number of times without success.

[MY SOLUTION]

After extensive reading of the logs and related forums, came up with a method that has worked for me in two windows machines already. Posted a batch file with the steps I used in https://github.com/jhvital/build-scipy-openblas.

The show_config method shows linkage to openblas libraries, and this was confirmed by comparing the runtimes of scripts against the scipy installed with conda install scipy, which installs the mkl package.

Still I'm not entirely sure whether the libraries were linked correctly. Feedback would be appreciated.

Joao Vital
  • 51
  • 4
  • 2
    Thanks for posting the script. I've tried following your script to install numpy+scipy without mkl on Windows but it still tries to install `mkl` when it gets to this line: `conda install -y blas numpy nose openblas`. I tried manually specifying `conda install numpy=1.16.2=py37_blas_openblash442142e_0` but it can't seem to find `openblas` when I do `np.show_config()`. How did you get it to install? – stevew Jul 27 '19 at 11:19
  • 1
    agree with @stevew. This doesn't work and wants to install mkl even if you have the nomkl meta package installed. – beginner_ Nov 12 '19 at 06:08
  • @stevew, @beginner_, I think conda now installs mkl by default, at the time I wrote the script it installed with openblas. Have you tried adding the flag `blas=*=openblas` to the end of that command line? – Joao Vital Nov 20 '19 at 10:48
0

The build instructions for SciPy is tested a number of times and it seems to be working.

You can find the instructions here http://scipy.github.io/devdocs/building/windows.html

Note that this is still a very delicate procedure and needs to be done very carefully. It still does not guarantee success thanks to Windows being Windows. However please take the time to report any issues should you encounter any at https://github.com/scipy/scipy.org/issues (notice it is not the scipy repo but but scipy.org repo)

percusse
  • 3,006
  • 1
  • 14
  • 28
0

I am not sure what's the error you have recieved when trying to get openblas & numpy using anaconda.

For openblas on windows using anaconda3 python 3.6v this works:

conda install -c menpo openblas

ref: https://anaconda.org/menpo/openblas

For numpy: it is :conda install numpy. if you are looking for a specific version of numpy use: conda install numpy=version_number

Pavan
  • 9
  • 3
  • 1
    This works but only for numpy. Doesn't work if you also need scipy or scikit-learn. The conda will want to install mkl again because as far as I know scipy builds on anaconda are only available for mkl. The only "solution" is to only use pip, assuming one find all the needed dependencies there for windows (unlikely) – beginner_ Nov 12 '19 at 06:11