Python on Windows, including rather complex science/math/statistics package builds? It should just work? Seriously, use a third-party distribution! There are plenty and they only exist as of the hassle you were observing.
While the other two answers are "okay", it would recommend another approach. Especially for the common Windows user the following approach is easy to maintain and easy to reproduce.
Let me quote from the SciPy website:
For most users, especially on Windows and Mac, the easiest way to
install the packages of the SciPy stack is to download one of these
Python distributions, which includes all the key packages:
[a list of Python distributions]
As you can see, people having experience with this topic tend to recommend using a third-party Python distribution.
I have so far used ActivePython, but Anaconda also appears to be a well-maintained project.
Whichever you take: it comes as a nice Windows installer and will give you a seamless Python experience.