2

I use it to implement neural networks. I prefer NumPy, because it is more convenient to prepare data with Python; however, I am concerned that NumPy is not as fast as c++ libraries.

Ami Tavory
  • 74,578
  • 11
  • 141
  • 185
Jason
  • 169
  • 1
  • 11
  • 3
    Rule of thumb: Python is slower than C++. – erip Jan 26 '16 at 23:54
  • Better rule of thumb: Python is definitely slower than C++ at computation. – erip Jan 26 '16 at 23:54
  • 1
    Buf numpy is implemented in C as Mike muler said below? – Jason Jan 27 '16 at 01:17
  • Python was implemented in C, but Python is slow. – erip Jan 27 '16 at 01:18
  • 1
    Really? Man, I don't think that is a good point. – Jason Jan 27 '16 at 01:21
  • 3
    It depends. If you have a well-tuned BLAS implementation, numpy will call into that, and that can be faster than Eigen. For instance on my machine I'm getting 200 G ops/sec on matrix multiplies using numpy, and only 160 G ops/sec using Eigen, with the reason being that my BLAS is better optimized (using OpenBLAS which uses openmp, vs using Eigen tensor library which relies on threadpools) – Yaroslav Bulatov Jan 28 '16 at 01:21
  • I tried mkl today, I thought it would be better than openBlas but not. How do you count the number of ops? – Jason Jan 28 '16 at 08:27
  • 5
    @erip Sorry, but I think that your comments, in the context of this question, are terribly misleading. Pure-Python's performance is nearly irrelevant for practical numerical work. – Ami Tavory Jan 28 '16 at 13:54
  • 1
    @AmiTavory You may disagree and that's fine. Perhaps I was thinking in the frame of "out of the box guarantees". I'm pretty sure numpy is still slower than Eigen/LAPACK _off the shelf_. Once you start fiddling, it may be comprable, but at that point OP is losing the "convenience factor" of Python. – erip Jan 28 '16 at 13:58
  • 1
    @erip I'm not even sure that I agree with your last comment (e.g., which Eigen/LAPACK C++: GSL, MKL? There's a huge variance), but your first comments are a different matter entirely. In any case, I guess we'll agree to disagree; all the best. – Ami Tavory Jan 28 '16 at 14:01

2 Answers2

7

NumPy is implemented in C. So most of the time you just call C and for some functionality optimized Fortran functions or subroutines. Therefore, you will get a decent speed with NumPy for many tasks. You need to vectorize your operations. Don't write for loops over NumPy arrays. Of course, hand-optimized C code can be faster. On the other hand, NumPy contains a lot of already optimized algorithms that might be faster than not so optimal C code written by less experienced C programmers.

You can gradually move from Python to C with Cython and/or use Numba for jit-compilation to machine or gpu code.

Mike Müller
  • 82,630
  • 20
  • 166
  • 161
6

I have to say that I think that the other answers here are missing things.

First, as @Mike Muller correctly points out, Python's numerical libraries have C or Fortran (or both) backends, so the performance of pure-Python is almost irrelevant (as opposed to the performance of the backend, which can be significant). In this respect, whether you're manipulating something like MKL through Python or C++ - hardly makes a difference.

There are two differences, though:

  • On the plus side for Python - it is interactive. This means that, especially in conjunction with something like the IPython Notebook, you can perform an operation and plot the result, perform another operation and plot the result, etc. It's hard to get this effect for exploratory analysis with a compiled language like C++ or Java.

  • On the minus side for Python - it, and its scientific ecosystem, handle multicores imperfectly, to say the least. This is a fundamental problem of the language itself (read about the GIL).

Ami Tavory
  • 74,578
  • 11
  • 141
  • 185
  • Your points are very reasonable to me. Also I want to know whether using a more deep learning specific toolkit like tensorflow,cntk, etc will lead to more efficient? – Jason Jan 29 '16 at 15:23
  • On the interactivity point: C++ now has a Jupyter kernel: https://github.com/jupyter-xeus/xeus – kjohnsen Jun 25 '21 at 23:50