3

In pandas, we have pd.rolling_quantile(). And in numpy, we have np.percentile(), but I'm not sure how to do the rolling/moving version of it.

To explain what I meant by moving/rolling percentile/quantile:

Given array [1, 5, 7, 2, 4, 6, 9, 3, 8, 10], the moving quantile 0.5 (i.e. moving percentile 50%) with window size 3 is:

1
5 - 1 5 7 -> 0.5 quantile = 5
7 - 5 7 2 ->                5
2 - 7 2 4 ->                4
4 - 2 4 6 ->                4
6 - 4 6 9 ->                6
9 - 6 9 3 ->                6
3 - 9 3 8 ->                8
8 - 3 8 10 ->               8
10

So [5, 5, 4, 4, 6, 6, 8, 8] is the answer. To make the resulting series the same length as the input, some implementation inserts NaN or None, while pandas.rolling_quantile() allows to compute the first two quantile values by a smaller window.

cs95
  • 379,657
  • 97
  • 704
  • 746
Roy
  • 880
  • 1
  • 12
  • 27

2 Answers2

7
series = pd.Series([1, 5, 7, 2, 4, 6, 9, 3, 8, 10])

In [194]: series.rolling(window = 3, center = True).quantile(.5)

Out[194]: 
0      nan
1   5.0000
2   5.0000
3   4.0000
4   4.0000
5   6.0000
6   6.0000
7   8.0000
8   8.0000
9      nan
dtype: float64

Center is False by default. So you need to manually set it to True for the quantile-calculation window to embrace the current index symmetrically.

Saeed
  • 1,848
  • 1
  • 18
  • 26
3

We could create the sliding windows with np.lib.stride_tricks.as_strided, implemented as a function as strided_app -

In [14]: a = np.array([1, 5, 7, 2, 4, 6, 9, 3, 8, 10]) # input array

In [15]: W = 3 # window length

In [16]: np.percentile(strided_app(a, W,1), 50, axis=-1)
Out[16]: array([ 5.,  5.,  4.,  4.,  6.,  6.,  8.,  8.])

To make it of the same length as the input, we could pad NaNs with np.concatenate or easier with np.pad. Hence, for W=3, it would be -

In [39]: np.pad(_, 1, 'constant', constant_values=(np.nan)) #_ is previous one
Out[39]: array([ nan,   5.,   5.,   4.,   4.,   6.,   6.,   8.,   8.,  nan])
Divakar
  • 218,885
  • 19
  • 262
  • 358
  • Marked as answer since it provides a solution to my question. Just curious, can your method extend to do the `min_periods` bit of `pandas.rolling_quantile()` (i.e. to create two additional strides at the beginning, each with fewer members)? @Divakar – Roy Dec 02 '17 at 07:56
  • 1
    @Roy Yup. Pad the input array with NaNs and use `np.nanpercentile`. – Divakar Dec 02 '17 at 08:25
  • Perfect. Thanks @Divakar. – Roy Dec 02 '17 at 17:19
  • 1
    This approach can lead to memory overflow for large input arrays. – Prokhozhii Jan 04 '21 at 20:24