6

When dealing with rolling windows, I wrote my functions in the way like list comprehension

[np.std(x[i:i+framesize]) for i in range(0, len(x)-framesize, hopsize)])]

Recently I discovered numpy.lib.stride_tricks.as_strided and found it is used widely for rolling windows (for example, this post), even though it is a "hidden" function.

In this issue concerning why stride_tricks.as_strided is undocumented, it's mentioned that

Intentionally! It's dangerous! It was just low-level plumbing to help implement broadcast_arrays().

Is there any advantage for stride_tricks.as_strided over the list comprehension or a for loop? I had a look at the source code of stride_tricks but gained little.

Thomas Baruchel
  • 7,236
  • 2
  • 27
  • 46
wsdzbm
  • 3,096
  • 3
  • 25
  • 28
  • The recent `split` question, http://stackoverflow.com/questions/38163366/python-split-list-in-a-specific-way may partly answer your question. The accepted answer uses a list comprehension, my answer striding. And I show how it can access values 'off-the-end'. – hpaulj Jul 04 '16 at 17:07

1 Answers1

3

From this post, we can use strided_app to basically get sliding views into the array and it also allows us to specify the hopsize/stepsize. Then, we simply use np.std along the second axis for the final output, like so -

np.std(strided_app(x, framesize, hopsize), axis=1)

Sample run for verification -

In [162]: x = np.random.randint(0,9,(11))

In [163]: framesize = 5

In [164]: hopsize = 3

In [165]: np.array([np.std(x[i:i+framesize]) \
            for i in range(0, len(x)-framesize+1, hopsize)])
Out[165]: array([ 1.62480768,  2.05912603,  1.78885438])

In [166]: np.std(strided_app(x, framesize, hopsize), axis=1)
Out[166]: array([ 1.62480768,  2.05912603,  1.78885438])

Being a view into the input array, these strided operations must be really efficient. Let's find it out!

Runtime test

Loopy approach -

def loopy_app(x, framesize, hopsize):
    return [np.std(x[i:i+framesize]) \
        for i in range(0, len(x)-framesize+1, hopsize)]

Timings -

In [185]: x = np.random.randint(0,9,(1001))

In [186]: framesize = 5

In [187]: hopsize = 3

In [188]: %timeit loopy_app(x, framesize, hopsize)
10 loops, best of 3: 17.8 ms per loop

In [189]: %timeit np.std(strided_app(x, framesize, hopsize), axis=1)
10000 loops, best of 3: 111 µs per loop

So, to answer the question on efficiency with strides, the timings should help prove a point there!

Community
  • 1
  • 1
Divakar
  • 218,885
  • 19
  • 262
  • 358
  • Hi @Divakar sorry I didn't make any tests done. this is an old post and I remember I used `strides` rather than list comprehension as the final solution. – wsdzbm Jan 16 '17 at 07:47