Python performance

Question

I have a question regarding the performance of my python program. The part which is written down is very essential and I already increased the performance with numpy. I would like to know if it is possible to make this part even faster? A 10x speed up would already be nice..

u = numpy.zeros((a**l, a**l))
re = numpy.zeros((a**l, a**l, a**l))
wp = numpy.zeros((a**l, 2))
...Some code which edits u,re and wp...
for x in range(N):
    wavg = numpy.dot(wp[:, 0], wp[:, 1])
    wp[:, 0] = 1.0/wavg*numpy.dot(u, numpy.multiply(wp[:, 0], wp[:, 1]))
    wp[:, 0] = numpy.tensordot(numpy.tensordot(re, wp[:, 0], axes=1), wp[:, 0], 
    axes=1)

Can you tell us anything about this code? Where are you running it, what does it do, what kind of input data do you have... — TigerhawkT3, May 10 '16 at 18:39
I'm voting to close this question as off-topic because it belongs on [codereview.se]. Questions on optimizing already-working code are off-topic on Stack Overflow. — MattDMo, May 10 '16 at 18:40
u is the mutation matrix. re is a recombination tensor. wp[:,0] is the current population vector and wp[:,1] the fitness vector. a is always 2 and l is usually 8. — HighwayJohn, May 10 '16 at 18:51
@linusg I think he wants to use numba for optimization. At least he added the numba-tag. But without using mathematical tricks this will be hard to optimize. — MSeifert, May 10 '16 at 19:12
[Cross-posted](http://codereview.stackexchange.com/q/128020/9357) on Code Review. — 200_success, May 10 '16 at 20:06

score 0 · Accepted Answer · edited May 23 '17 at 10:28

I made a great experience with Cython (Another thing than CPython...). You can make plain C code out of your program, compile it to an extension and just run it from another Python file. See this question from me for more information on building an extension including numpy: How to create a .pyd file?.

Sample:

Assuming you have your code compiled to an extension (.pyd on Windows, .so on every other platform), e.g. myext.pyd and a function myfunc() in it, you can run it like this:

import myext
myext.myfunc()

The Python Interpreter will run this little code snippet, load the extension (which is bytecode made from C-code, will be fast!) and run any function much faster than normal Python code.

You will need to install a compiler on Windows, gcc is already included in most Linux/Unix systems. There's a special Compiler package for Python 2.7, but this dies not work for Python 3.x. You can use MinGW on Windows, a Port of the gcc which works fine for me.

Sometimes Cython will generate 15k lines C code from about 300 lines Python Code, don't wonder about that :)

Feel free to ask if you have some more questions or need a more detailed description how to do that. Building a working Python extension is not that trivial...

Hope this helps!

I made some edits, may be useful information – linusg May 11 '16 at 13:41 — linusg, May 11 '16 at 13:41

Python performance

1 Answers1