This can't be vectorised and we need to loop through the rows. I'm wondering if this can be done effectively in polars without casting. I see in the polars documentation for polars.DataFrame.rows it says:
Row-iteration is not optimal as the underlying data is stored in columnar form; where possible, prefer export via one of the dedicated export/output methods.
In pandas/numpy, the fastest way I can imagine is to use numba, roughly like this:
import numba as nb
import numpy as np
@nb.jit(nopython=True)
def exponential_sum(signal, decay, initial_value=0):
n = len(signal)
exp_sum = np.zeros(n)
exp_sum[0] = signal[0]
for i in range(1, n):
exp_sum[i] = exp_sum[i-1] * decay[i] + signal[i]
return exp_sum
where decay = np.exp(times.diff() * alpha) (alpha controls the decay rate and the half-life).
I have tried casting to numpy and using numba as mentioned, but I'm wondering if there's a in-polars approach that is performant, and if not whether: it is just not implemented or it is a problem which polars is not well suited for due to the storage format.
