0

Dear stackoverflow community!

In a previous stackoverflow question, I mentioned that python's np.fft.fftn() routine seems somehow slow compared to MATLAB, provided that the datacubes are rather big (grids of dimension 512x512x1921, datatype float) (see Comparatively slow python numpy 3D Fourier Transformation). I think that MATLAB adopts the FFTW algorithm and could therefore be faster (~5s compared to ~185s (time.time())), so I was suggested to try pyFFTW for a time reduction.

The problem now is that at my work place python packages are implemented via anaconda for a large number of computers and the pyFFTW package cannot be easily integrated therein. There's somehow also a problem that long datatypes are not recognized and therefore compilation does not work at all. pyFFTW also conflicts with the internal FFTW implementation. Even if somehow installed, it would be overriden by the next update of the system.

I'm however not sure whether the different algorithm alone would explain the difference in computation time. As already written in the previous questions, I really need these FFTs for my work.

Another issue concerns the striding of the output array of np.fft.fftn(), which is switched to FORTRAN structure automatically (which again is the opposite of the default). This causes low performace when operating on the output in combination with C-strided grids (see Python numpy.fft changes strides).

So as a follow-up to my original questions, I want to ask you:

(MAIN) What other reasons might there for python to be so much slower? What can be done about it? I'd like to stay with python if possible and rather not switch to MATLAB just because of such things...

(SIDE) Is there any keyword to preserve striding? Using scipy is not a good option and copying the array to a new one to get the strides correctly also seems an unnecessarily complicated step requiring additional computation time.

Thanks for the help!

Community
  • 1
  • 1
bproxauf
  • 1,076
  • 12
  • 23
  • Profile your code, find the bottlenecks and post them as a question. Then people can help to optimise it in Python. As a last resort you can always call C/C++ code from Python. – DBedrenko Oct 25 '16 at 09:49
  • How did you install pyFFTW? setup.py, wheel, conda-forge? Also, do you have the MKL libraries installed on anaconda? Those can improve FFT performance considerably. Finally, have you tried using `scipy.fftpack` in place of `numpy.fft`? Also, which operating system are you using? – TheBlackCat Oct 25 '16 at 21:15
  • scipy.fftpack has similar performace to numpy.fft as far as the answers in previous comments went. I'll ask for the exact installation method as it's a grid/cluster architecture with software updates and installation done by the IT department... – bproxauf Oct 26 '16 at 13:43

0 Answers0