0

I ask this question the third time because it was not clear enough and I hope it is more understandable now:

I want to iterate a function f n times without using a loop

The function f returns (10,) vectors and runs trough n iterations. The output should be an (n, 10) array. I chose the random function as a simple example for a more complex function f. The parameter f(x) was necessary for my attempt of vectorizing, other than that, it is obviously useless.

Below are some examples which are all supposed to do the same thing. But none of them is efficient.

I was told that numpy.vectorize is generally not efficient because it's a conveniance function. On the other hand I was told that numpy offers many solutions to do exactly what I want to do.

So does anyone have an idea what numpy function or procedure is more efficient than vectorize?

TIA


import numpy as np
import time


x=0
n_iterations = 100000
n_cols = 10


def f(x):
    return np.random.rand(1,n_cols)


# Speed comparison: 
start_time = time.time()
arr = np.random.rand(n_iterations,n_cols)
print("Run time: original function = {}".format(time.time() - start_time))
print(arr.shape)


# 1) for loop
start_time = time.time()
results1 = np.empty([n_iterations,n_cols]) 
for i in range(n_iterations):
    results1[i,] = f(x)

print("Run time: for loop = {}".format(time.time() - start_time))
print(results1.shape)
print(results1[0].shape)
#print(results1)

# 2) Faster but returns wrong array format
start_time = time.time()
f_vectorize = np.vectorize(f, otypes=[np.ndarray])
a = np.arange(n_iterations)
results2 = f_vectorize(a)

print("Run time: vectorize/arrange = {}".format(time.time() - start_time))
print(results2.shape)
print(results2[0].shape)
#print(results2)


# 3) vectorize/signature/arrange: 
start_time = time.time()
f_vectorize = np.vectorize(f, signature='()->(n)', otypes=[np.float32])
a = np.arange(n_iterations)
results3 = f_vectorize(a)

print("Run time: vectorize/signature/arrange= {}".format(time.time() - start_time))
print(results3.shape)
print(results3[0].shape)
#print(results3)
greg2021
  • 1
  • 1
  • 1
    Numpy can't magically make some complex function faster. You might want to ask on SE/Code Review for advice on optimising a specific function, but it isn't really appropriate here on Stack Overflow. See [how to ask](https://stackoverflow.com/help/how-to-ask). – Felix Nov 03 '20 at 12:18
  • Just to be clear: In your actual code, is ``f`` your own function that you can readily modify or re-implement, or is it some black-box that just returns an array? – MisterMiyagi Nov 03 '20 at 12:31
  • @MisterMiyagi: it justs returns an array. – greg2021 Nov 03 '20 at 12:38
  • Then I don't think you can do anything "better". As long as ``f`` might do any of the regular Python shenanigans, ``numpy`` has no way of magically ignoring these constraints. – MisterMiyagi Nov 03 '20 at 12:49
  • And how about indexing as in meth5 in this thread? https://stackoverflow.com/questions/53155749/replace-elements-in-numpy-array-avoiding-loops – greg2021 Nov 03 '20 at 13:28
  • 1
    You already found the optimal `numpy` way, the 2d `rand`. This uses the existing compiled `numpy` methods to produce the whole array without a Python level iteration. As long as your `f` function is a Python function that has to be called 100000 times, there isn't much you can do to change that. `iteration <==> loops`! – hpaulj Nov 03 '20 at 17:01

0 Answers0