1

I'm trying to optimize my code works heavily with numpy functions. I created a simple version for here.

#pythonic.py
import numpy as np

def numpyWithPython():
        a = np.random.randint(10, size=5000) -5
        b = a[a>0]
        c = np.cumprod(b)
        d = np.diff(c, prepend=0)
        e = np.multiply(d,b)
        f = e.copy()
        f[f >1] =2
        g = np.round(f*5000)

        if np.sum(g)>0:
                return np.sum(g)
        else:
                return 20

So basically the whole function works with numpy. Then I compiled it to Cython, I tried my best to optimize but I'm not sure about it, anyway, code looks like this:

#Cythonic.pyx
import numpy as np
cimport numpy as np

# cython: boundscheck=False
# cython: cdivision=True
# cython: wraparound=False

cpdef int numpyWithCython():
        cdef np.ndarray[long, ndim=1, mode='c'] a = np.random.randint(10, size=5000) -5
        cdef np.ndarray[long, ndim=1, mode='c'] b = a[a>0]
        cdef np.ndarray[long, ndim=1, mode='c'] c = np.cumprod(b)
        cdef np.ndarray[long, ndim=1, mode='c'] d = np.diff(c, prepend=0)
        cdef np.ndarray[long, ndim=1, mode='c'] e= np.multiply(d,b)
        cdef np.ndarray[long, ndim=1, mode='c'] f=e.copy()
        f[f >1] =2
        cdef np.ndarray[long, ndim=1, mode='c'] g =np.round(f*5000)

        if np.sum(g)>0:
                return np.sum(g)
        else:
                return 20

And let's compare them

from pythonic import numpyWithPython
from Cythonic import numpyWithCython
import timeit

py = timeit.timeit(numpyWithPython, number=1000)
print("Python : ",py)

cy = timeit.timeit(numpyWithCython, number=1000)
print("Cyhton : ",cy)

Than I get evet negative improvement, which is unexpected...

>>>Python :  0.22221166321079613
>>>Cyhton :  0.23356160436067247

Is my optimization for Cython wrong or in this case Numpy is already fast so Cython makes no sense?

Should I go with native python arrays to speed up, or what can I do to go faster?

Emir Atakan Yılmaz
  • 33
  • 4
  • For who might have an interest in speeding, I started with pandas.DataFrame, then I had ~2X better result with pandas.Series, than Numpy version was even ~4X even faster. I can share also share setup.py to show how did I do compiling to Cython – Emir Atakan Yılmaz Aug 06 '19 at 11:44
  • 1
    Most numpy functionality is highly optimized compiled c-code. I would be very surprised if you're be able to speed up neyond that. – jmetz Aug 06 '19 at 11:46
  • Look for `annotate=True` on https://cython.readthedocs.io/en/latest/src/tutorial/cython_tutorial.html this will tell you how well your Cython code transforms to C and if there are bottlenecks, see also https://stackoverflow.com/questions/48272969/cython-understanding-what-the-html-annotation-file-has-to-say – Joe Aug 06 '19 at 12:39
  • 4
    All you've really done is added a few pointless type checks where Cython makes sure that the arguments are Numpy arrays - that's why it's slower. – DavidW Aug 06 '19 at 14:11
  • 1
    If you need to speed the code up, you need to ditch the numpy function calls. You do not want to create 6 intermediate numpy arrays; allocating the memory for each one and storing all of that data will slow you down. Try to operate in a for loop on each element and do all of the math operations 1 element at a time. – CodeSurgeon Aug 06 '19 at 17:02

0 Answers0