0

I have installed python 3.5.2 through the official installer found here and I cloned numpy and scipy (see here). I have Intel parallel studio 2017 installed, so that I have Intel's versions of blas and lapack (they are in the Math Kernel Library (MKL)), and I understand these are needed to build numpy and scipy (do I miss something else needed ?) and I would like to build numpy and scipy from source, without using anaconda or wheel or whatever. I typically don't want to resort to things like

Installing SciPy with pip

(even if I had the same pip issue as described there) and I want to build libraries myself because I want to have binaries and libraries optimized for my system. (I will be grateful for answers respecting that.)

How should I do ?

Remarks. I am under win10 64bits, have visual studio 2013 and 2015 and intel parallel studio 2017 with last intel MKL version. I also have python 2.7.12 and for now I would also be satisfied with being able to build numpy and scipy with python 2.7.12. (I may be wrong but I don't think that the right procedure to build numpy and scipy with with python and intel MKL's blas and lapack strongly depends on the fact that one uses python 2.7.12 or 3.5.2.)

Community
  • 1
  • 1
Olórin
  • 3,367
  • 2
  • 22
  • 42
  • So why no anaconda and why no pip? The classic non-pip approach is calling setup.py. But this will probably be a lot of pain on windows (numpy less painful than scipy). – sascha Nov 12 '16 at 13:04
  • 1
    I found this : https://software.intel.com/en-us/articles/building-numpyscipy-with-intel-mkl-and-intel-fortran-on-windows I think it is going to solve my problems for numpy at least – Olórin Nov 12 '16 at 13:19
  • Intel free distribution of Python is really a great choice – Saullo G. P. Castro Nov 12 '16 at 14:16
  • Why no anaconda and why no pip ? Because even on windows I try to stick as much as possible to what I do on *nix : build everything I can myself. – Olórin Nov 12 '16 at 18:31

1 Answers1

1

For Windows, I prefer to download the wheel (.whl) files from Christoph Gohlke's LFD page. He builds using the MKL math libraries, and tracks updates as they are released (and tracks pre-release versions for important packages). I can build these myself, but it makes more sense to me to use the Christoph's great work instead.

cco
  • 5,873
  • 1
  • 16
  • 21
  • While i also prefer the way of using prebuilt-stuff (if it's high-quality; i have to admit that i much much prefer anaconda), this does not answer the question as he explicitly states that he want to compile from the sources. I asked about the why, but no reaction though. – sascha Nov 12 '16 at 13:42
  • That's fair. I wrote this as an answer because I thought a pointer to Christoph's work would be more useful to people trying to get MKL numpy & scipy working that aren't set on building it, but didn't know of any alternative. Wheels make things much easier - I have a local cache of LFD wheels for Windows, but I have a complete Jupyter/Numpy/Pandas stack on my Mac using only `pip` (the MacOS `.whl` files bind to the Apple fast math libraries too). – cco Nov 12 '16 at 13:51
  • Great that these binaries work well for you. I switched to anaconda for a long time now as it's much more as only binaries. It nicely adds tools for environment-encapsulation as well and a nice pip-like installer. And of course these builds are based on Intel's MKL (maybe not the ones for Mac, but i'm not sure). – sascha Nov 12 '16 at 13:54
  • 1
    All true. When I started using Numpy, Anaconda didn't use MKL on Windows, but now they do (for Mac too, I think). We have the luxury of multiple good choices. Cheers! – cco Nov 12 '16 at 13:57
  • While the "non-solution" that you propose (and that you could have rather proposed as answer to similar questions not stressing "no anaconda no pip") has the advantage to be "coherent" indeed but is for sure non-optimized regarding your system (OS etc etc), I prefer even on windows to stick as much as possible to what I do on *nix : build everything I can myself so that I can have as much as possible binaries/libraries optimized for my system. – Olórin Nov 13 '16 at 13:16
  • 64-bit windows is a very simple ecosystem as far as the OS goes, and the MKL libraries (which I don't have a license to build with, but can use if distributed by someone who does) used by Christoph are better than I could build. For other systems (where building isn't nearly as painful as Windows, other choices make more sense. – cco Nov 13 '16 at 21:48
  • So you have the same intel processor as the 64bits windows Christoph built the libraries under ? Because optimization goes up to the processor as well, so that your choice (keyword "your") makes more sense to you indeed, but obviously not in general. That's why I am asking my question. – Olórin Nov 13 '16 at 23:07
  • I'm under the impression that processor-specific optimizations are taken at runtime, not at compile time (I could be wrong about that); in any case, I don't have BLAS code set up on Windows, so even a suboptimal MKL is (much) better than no fast math code. – cco Nov 14 '16 at 02:29