You can proceed with installing numpy or scipy using pip, after you install lapack and blas, which are system libraries. It shouldn't be very hard, but depends on you OS.
For RedHat/CentOS/Fedora this could be done with:
yum install lapack lapack-devel blas blas-devel
The packages can be found e.g. in CentOS base repository.
However, the scikit-learn website says as follows:
We don’t recommend installing scipy or numpy using pip on linux, as this will involve a lengthy build-process with many dependencies. Without careful configuration, building numpy yourself can lead to an installation that is much slower than it should be. If you are using Linux, consider using your package manager to install scikit-learn. It is usually the easiest way, but might not provide the newest version. If you haven’t already installed numpy and scipy and can’t install them via your operation system, it is recommended to use a third party distribution.
Package managers are usually yum or apt-get and again on RedHat/CentOS/Fedora you can skip using pip and install this way:
yum install scipy
Third party distributions mentioned above are things like anaconda or Python(x,y).