Performance optimizations of Python with Cython

Question

I was learning Python and came across something called Cython helping to gain performance improvements. Later, I found this talk https://www.youtube.com/watch?v=_1MSX7V28Po presented by the developer of Instagram and during his talk he said that Cython is truly helpful in terms of Python optimizations. But briefly mentioned that if you just add Cython is good but if you add types annotations for your python code. You can achieve much better performance. So, the question is Is it true that by just adding cython we can get performance boost BUT if we add TYPE ANNOTATIONS for python code, then performance boost will be much more better?

The typed variables in Cython are distinct from Python type annotations — juanpa.arrivillaga, May 17 '20 at 05:56
Using Cython typed variables in Cython will provide a performance benefit. `typing`-style type annotations provide no performance benefit; at worst, they may actually slow your code down. — user2357112, May 17 '20 at 05:58
@juanpa.arrivillaga, I mean if we use type system of cython, will it give us additional and better performance boost compared to just adding cython and not using type system of it in our python code? — SAM, May 17 '20 at 06:03
Spend time reading the `cython` docs. Tutorials and blogs only give you suggestions, not the real deal. And at the start, don't throw cython at all of your code. Focus on some functions that really need the speed improvement. You can loose a lot of Python's flexibility if you switch to a compiled version too soon. — hpaulj, May 17 '20 at 07:40
Some time ago I have analysed what is responsible for the speed-up with Cython: https://stackoverflow.com/a/46723823/5769463, in a nutshell: Python objects for int/float have too much overhead and are main bottleneck. — ead, May 17 '20 at 08:23

score 0 · Answer 1 · answered May 17 '20 at 15:10

Taking a look at the documentation I think we can answer your question.

However, for performance critical code, it is often helpful to add static type declarations, as they will allow Cython to step out of the dynamic nature of the Python code and generate simpler and faster C code - sometimes faster by orders of magnitude.

Example without type annotations:

def f(x):
    return x ** 2 - x


def integrate_f(a, b, N):
    s = 0
    dx = (b - a) / N
    for i in range(N):
        s += f(a + i * dx)
    return s * dx

Simply compiling this in Cython merely gives a 35% speedup. This is better than nothing, but adding some static types can make a much larger difference.

Example with type annotations:

def f(double x):
    return x ** 2 - x


def integrate_f(double a, double b, int N):
    cdef int i
    cdef double s, dx
    s = 0
    dx = (b - a) / N
    for i in range(N):
        s += f(a + i * dx)
    return s * dx

Since the iterator variable i is typed with C semantics, the for-loop will be compiled to pure C code. Typing a, s and dx is important as they are involved in arithmetic within the for-loop; typing b and N makes less of a difference, but in this case it is not much extra work to be consistent and type the entire function.

This results in a 4 times speedup over the pure Python version.

So it appears that just using Cython does provide and performance boost, and that adding type annotations provides even more of a performance boost.

Performance optimizations of Python with Cython

1 Answers1