How to let PyPy recognize linear algebra (Intel MKL, OpenBLAS) libraries? Environmental variables issue?

Question

I have a working Python 3.6 Version installed and wanted to try out PyPy for a heuristic algorithm. I installed PyPy using this guide: How to use PyPy on Windows? and got it to run and also install e.g. the openpyxl module via: pypy3 -m pip install openpyxl

However, when trying to install numpy or pandas I get the following error messsage:

"error: Microsoft Visual C++ 14.1 is required. Get it with Microsoft Visual C++ Build Tools": https://visualstudio.microsoft.com/downloads/"

I tried all of the following solutions: Microsoft Visual C++ 14.0 is required (Unable to find vcvarsall.bat) namely:

1. Updated Setuptools pypy3 -m pip install --upgrade setuptools
2. Installed various kinds of Microsoft Visual Studio/Build Tools as mentioned in the comments of that question.
3. Tried to install numpy using the binary only option pypy3 -m pip install --only-binary :all:numpy
4. Tried updating setuptools and installing numpy from the VS command prompt.

None of these worked for me. The best I came up with was that it might have to do with the environment variables. In the PyPy doc (http://doc.pypy.org/en/latest/windows.html) it says:

"The installation will set the VS140COMNTOOLS environment variable, this is key to distutils/setuptools finding the compiler"

I could not find this variable in the Systemvariables or Uservariables. So I tried making a new SYSTEMVARIABLE with name = VS140COMNTOOLS and Value = C:\Program Files (x86)\Microsoft Visual Studio\2019\BuildTools\Common7\Tools. That is the path where on my machine the VsDevCmd.bat is located, which I think is responsible for setting the environment variable during installation of Microsoft Visual Studio.

This did change my error message but only made it worse/longer. Also, when I type python or pypy3 in my command prompt I get the following:

!https://i.stack.imgur.com/vcGmJ.jpg

Which according to this post What is the difference between these two lines, MSC v.1900 64 bit (AMD64) and MSC v.1914 32 bit (Intel) show that I have a C++ compiler installed?

My questions are:

1. Does the Name of the environmental variable ("VS140COMNTOOLS") matter? I assume it is for VisualStudio14 but the Value leads to VisualStudio19.
2. Is it right to use a SYSTEMVARIABLE and not a USER VARIABLE to point to the compiler?
3. If I need Microsoft Visual C++ 14.1, are/should all newer versions also be feasible?
4. Is my environmental variable actually pointing to the right directory? What needs to be in such a directory to make PyPy recognize a C++ compiler?
5. Why do I have to type pypy3 -m pip install <Package>? instead of pypy3 pip install <Package> which only results in a "FilenotFoundError"?
6. How do i get PyPy to recognize the compiler and install numpy or pandas correctly?

EDIT 1

Using the "x86 Native Tools Command Prompt for VS 2019" and trying pypy3 -m pip install numpy did not work. I also found this question how can I install numpy on pypy on my 64 bit computer running Windows 64 bit? and since I could not find the respective environmental variable: I set it using set VS140COMNTOOLS=C:\Program Files (x86)\Microsoft Visual Studio\2019\BuildTools\VC\Auxiliary\Build" and tried both: pypy3 -m pip install numpy and pypy3 -m pip install numpy -v. Sadly, that also did not resolve the issue and the error message still is: and later on saying that no Microsoft Visual C++ 14.1 is installed.

The error message points out it does not find several libraries in my PyPy/libs folder. Does anyone know what I need to install that it finds these libraries?

@mattip Speed does not really matter in my case, since I actually don't use numpy for calculations. However I use pandas for data storage which requires numpy to be installed first.

Is it easier to install PyPy in a virtual environment, or will I face the same issues there?

Answer 1

The prompt you get when you run PyPy indicates which compiler it was built with, not which compiler is found. Even if PyPy could find the compiler it is looking for, building NumPy locally on windows will result in a slower version of NumPy. NumPy really needs a linear algebra back end like OpenBLAS or Intel's MKL, and getting those installed is tricky. Now for your questions:

1. You could start up the Visual Studio Command Line for 32 bit compilers and then run PyPy in that DOS window, I am pretty sure then NumPy will build (but the result will be slower than a vnedor-provided binary wheel).

2. The end result is exactly the same. SystemVariable will set that for all users, UserVariable only for you.

3. Yes, probably. I don't have 2019, only 2017.

4. PyPy uses the services provided by setuptools and distutils just as regular CPython does. So the answers to that are the same as the answers you find for CPython.

5. The -m means "load this module" so you are loading the pip module and running its code. Without the -m you are running a file named pip which does not exist on your path

6. The path forward is to make PyPy a first class citizen in the python world. Please urge the projects themselves (pandas, numpy, scipy ...) to provide wheels for PyPy, as well as asking Anaconda to provide a PyPy variant.