2

In my dataframe, I have to repeat the same function 300000 times, each time adding 1 to a variable var. The easy way for me to do this is by using a for loop

for var in range(0,300000): 
   function(var)

Is there a faster way to iterate this function? (I don't really know if is important in this case, but my function is doing its thing over a np.array)

Nycolas Mancini
  • 153
  • 7
  • Does this answer your question? [Most efficient way to map function over numpy array](https://stackoverflow.com/questions/35215161/most-efficient-way-to-map-function-over-numpy-array) – michaeldel Apr 25 '20 at 18:13
  • Have a look at this example. Assuming you are just adding 1 to a Numpy array. If the array is named data, you should just be able to do data = data + 1 https://stackoverflow.com/questions/5754571/sum-one-number-to-every-element-in-a-list-or-array-in-python – Sri Apr 25 '20 at 18:13
  • the function in step n was executed on the result of the function in n-1? – ansev Apr 25 '20 at 18:22
  • It depends on what `function` is doing. If it's something that can be [vectorized](https://docs.scipy.org/doc/numpy/reference/generated/numpy.vectorize.html), or is already vectorized, then you can do `function(np.arange(300000))`. But if not, then you gotta do a loop, although again depending on the function, a [list comprehension](https://leadsift.com/loop-map-list-comprehension/) might be faster. – stevemo Apr 25 '20 at 18:56
  • @stevemo, don't suggest `np.vectorize`. It will not run faster (read its disclaimer). – hpaulj Apr 25 '20 at 20:18
  • Your `function` only gets a scalar integer. What's this `array` it's "doing its thing over". And where's the `dataframe` in the story. Lots of vagueness in this question. But the short answer is, if `function` is a python function that only works with one scalar at time, the you have to call for each of those 30000... times. The trick to doing things faster in `pandas` and `numpy` is to write functions that use fast compiled methods that work with *whole* arrays and frames (not just with one integer at a time). – hpaulj Apr 25 '20 at 20:22

0 Answers0