3

I have some function f(x,y) which I have vectorized with numpy.vectorize command. I have made some grid values of x and y for which I want the function to be evaluated. My program looks like this:

from numpy import vectorize,meshgrid, linspace

@vectorize
def f(x,y):
    pass

x = linspace(0,10)
y = linspace(0,10)
X, Y = meshgrid(x, y)
Z = f(X,Y)

When I look at the system monitor on evaluation time (for example with htop on Ubuntu) I see that only one core is being used. What are the options to get most juice of system on such computation?

Jānis Erdmanis
  • 389
  • 3
  • 13
  • what does your f(x,y) do? instead of using @vectorize, it is better to implement your function so that it can handle ndarray arguments, since numpy.vectorize is bad – usethedeathstar Apr 29 '14 at 13:23
  • The python interpreter runs on only one core, though `numpy` can use a multi-core setup (see e.g. http://stackoverflow.com/questions/5991014/numpy-on-multicore-hardware). A simple way to make use of multiple cores is to use something like the `multiprocessing` module. – jmetz Apr 29 '14 at 13:23
  • Generally the `f(x,y)` integrates some other function `g(T,x,y)` along T with `scipy.integrate.quad`. Why numpy.vectorize is bad? It makes coding much easier! – Jānis Erdmanis Apr 29 '14 at 15:45
  • Okay how to implement vectorize function in order to use multiple cores? – Jānis Erdmanis Apr 29 '14 at 15:52

1 Answers1

3

Using numpy.vectorize is not usually a good idea. It doesn't parallelize anything, and in fact the performance is generally not even very good because of the way it works (as explained in its documentation). So instead, you need to work on making your f(x,y) function vectorized internally, or implement it as a module in C or C++ or Fortran etc.

If you want to truly use multiple cores in Python, your best bet is either to use the threading module if your function is implemented in C, or multiprocessing if you're trying to run native Python (including NumPy).

Adobe
  • 12,967
  • 10
  • 85
  • 126
John Zwinck
  • 239,568
  • 38
  • 324
  • 436
  • (+1 for what's in the answer.) If `numexpr` can handle the function `f(x, y)`, that can be the easiest way of using several cores. – ev-br Apr 29 '14 at 13:36