1

I'd like to perform a function on a vector/matrix in the way a running total does.

Simply said 

  perform the function for each row 
    from the first row up and including the current one

I found various solutions for the running totals, basically with the cumsum function.cumsum1 dplyr cumsum2

But already the mean function did not work in the way I look for. And the rle also works only for whole vector.

Example

> df <- data.frame(value = df <- data.frame(value = c(1, 0, 0, 1, 0, 0, 0, 0, 0, 0, 1, 1, 0, 1, 0, 1, 1, 0, 1, 1)))
> df$csum <- ave(df$value, FUN=cumsum)
> df$mean <- ave(df$value, FUN=mean)
> df
   value csum mean
1      1    1 0.45
2      0    1 0.45
3      0    1 0.45
4      1    2 0.45
5      0    2 0.45
6      0    2 0.45
7      0    2 0.45
8      0    2 0.45
9      0    2 0.45
10     0    2 0.45
11     1    3 0.45
12     1    4 0.45
13     0    4 0.45
14     1    5 0.45
15     0    5 0.45
16     1    6 0.45
17     1    7 0.45
18     0    7 0.45
19     1    8 0.45
20     1    9 0.45

But I would like to get:

     value     csum     mean     run_mean
 1       1        1     0.45     1
 2       0        1     0.45     0,5
 3       0        1     0.45     0,333333333
 4       1        2     0.45     0,5
 5       0        2     0.45     0,4
 6       0        2     0.45     0,333333333
 7       0        2     0.45     0,285714286
 8       0        2     0.45     0,25
 9       0        2     0.45     0,222222222
10       0        2     0.45     0,2
11       1        3     0.45     0,272727273
12       1        4     0.45     0,333333333
13       0        4     0.45     0,307692308
14       1        5     0.45     0,357142857
15       0        5     0.45     0,333333333
16       1        6     0.45     0,375
17       1        7     0.45     0,411764706
18       0        7     0.45     0,388888889
19       1        8     0.45     0,421052632
20       1        9     0.45     0,45

Now I know that I could use cumsum and division to solve the mean-challenge. But I'd like to a general approach to solve something like the rle

> df$rle <- ave(df$value, FUN=rle)
> df
   value csum mean                             rle
1      1    1 0.45 1, 2, 1, 6, 2, 1, 1, 1, 2, 1, 2
2      0    1 0.45 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1
3      0    1 0.45 1, 2, 1, 6, 2, 1, 1, 1, 2, 1, 2
4      1    2 0.45 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1
5      0    2 0.45 1, 2, 1, 6, 2, 1, 1, 1, 2, 1, 2
6      0    2 0.45 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1
7      0    2 0.45 1, 2, 1, 6, 2, 1, 1, 1, 2, 1, 2
8      0    2 0.45 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1
9      0    2 0.45 1, 2, 1, 6, 2, 1, 1, 1, 2, 1, 2
10     0    2 0.45 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1
11     1    3 0.45 1, 2, 1, 6, 2, 1, 1, 1, 2, 1, 2
12     1    4 0.45 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1
13     0    4 0.45 1, 2, 1, 6, 2, 1, 1, 1, 2, 1, 2
14     1    5 0.45 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1
15     0    5 0.45 1, 2, 1, 6, 2, 1, 1, 1, 2, 1, 2
16     1    6 0.45 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1
17     1    7 0.45 1, 2, 1, 6, 2, 1, 1, 1, 2, 1, 2
18     0    7 0.45 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1
19     1    8 0.45 1, 2, 1, 6, 2, 1, 1, 1, 2, 1, 2
20     1    9 0.45 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1
>

Any suggestions for a newbee?

EDIT: Made the example reproduceible (constants instead of sample)

Community
  • 1
  • 1
Kurt Ludikovsky
  • 682
  • 1
  • 6
  • 21
  • Not sure what the ave calls are supposed to be doing: they are for disjoint categories and they need three arguments: For the requested running mean function why not . `function(x) {cumsum(x)/(1:length(x) )}` – IRTFM Sep 01 '16 at 06:39
  • 1
    Note: Please use `set.seed` so we can check our results against your expected output – Sotos Sep 01 '16 at 06:40
  • @42: That's what I meant, that I know I can calculate the mean by division. But this is not what I am after. – Kurt Ludikovsky Sep 01 '16 at 07:07
  • Clarification: I am after a generic function not a solution for the `mean`. Please see and take the `rle` example – Kurt Ludikovsky Sep 01 '16 at 07:10
  • Depending on what you want to do, you could formulate the calculation in a cumulative way, so that f(1:(n+1)) = F(f(1:n), n+1) and then simply use a functions from the apply family to apply F. – snaut Sep 01 '16 at 07:26
  • It's quite simple [to define a `cumapply` function](https://stat.ethz.ch/pipermail/r-help//2009-September/404848.html). However, is would call the function for each row, which is not efficient. Depending on the specific case there are often much better solutions as the answer demonstrates. – Roland Sep 01 '16 at 07:27
  • @Roland: the `cumapply`function sounds good. I agree that this might bei inefficient, that's the reason for my question here. As the function assume the `rle`-function on binary values looking for the lengths.Mitght the `dplyr`-function ` slice`be of any help to improve speed? – Kurt Ludikovsky Sep 01 '16 at 08:45
  • @snaut: I am not sure if this could be a solution, but maybe - especially with `rle`. How would I be able to access a (n-1) value from an n-th-element in the apply function? Any hint's, as my current understanding is that the apply is working for each element independently. – Kurt Ludikovsky Sep 01 '16 at 08:52
  • I'm not sure what your expected output from applying `rle` cumulatively would be. – Roland Sep 01 '16 at 09:28
  • @KurtLudikovsky you'd have to apply to seq_along(1:nrow) or something similar, but the more I think about it the worse the idea seems ;) – snaut Sep 01 '16 at 10:26
  • @Roland:the term cummulative is somewhat wrong here and just cam via the running total and `cumsum`. I don't know how to better explain. I'd simply like to perform a function for each element from the start. As with the ´apply` functions - as understood them - where one performs a function on each element, i'd like to perfom the function on all elements from x(1.n). Technically spoken: assum that each element representat a sample on a digital line. the ´rle` gives me the lengths of high and low (number of counts) from which I can furtheron calculate means and deviations of lengths. – Kurt Ludikovsky Sep 01 '16 at 16:05
  • @snaut: As I am new to R I simply asked if there is an elegant solution to this. As a seasoned 'old-style' programmer I know that this can be done with loops. But as I am lazy I'd like to always get a simpler solution. :-) – Kurt Ludikovsky Sep 01 '16 at 16:08
  • I was also checking if the dplyr `slice` could be of some help, but have some troubles getting it running. `install.package` created an error first time, after reinstalling it still didnt find the function. Need to check tommorrow. – Kurt Ludikovsky Sep 01 '16 at 16:11

1 Answers1

2

You can try

df$run_mean <- cumsum(df$value)/seq_len(nrow(df))
#> head(df)
#  value csum mean  run_mean
#1     1    1 0.45 1.0000000
#2     0    1 0.45 0.5000000
#3     0    1 0.45 0.3333333
#4     1    2 0.45 0.5000000
#5     0    2 0.45 0.4000000
#6     0    2 0.45 0.3333333

Basically it is the cumulative sum of the value divided by the row number.

As pointed out by @akrun the dplyr package provides a function cummean() that calculates just that. Hence an alternative could be:

df$run_mean <- dplyr::cummean(df$value)

data

df <- structure(list(value = c(1L, 0L, 0L, 1L, 0L, 0L, 0L, 0L, 0L, 
0L, 1L, 1L, 0L, 1L, 0L, 1L, 1L, 0L, 1L, 1L), csum = c(1L, 1L, 
1L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 3L, 4L, 4L, 5L, 5L, 6L, 7L, 7L, 
8L, 9L), mean = c(0.45, 0.45, 0.45, 0.45, 0.45, 0.45, 0.45, 0.45, 
0.45, 0.45, 0.45, 0.45, 0.45, 0.45, 0.45, 0.45, 0.45, 0.45, 0.45, 
0.45), run_mean = c(1, 0.5, 0.333333333333333, 0.5, 0.4, 0.333333333333333, 
0.285714285714286, 0.25, 0.222222222222222, 0.2, 0.272727272727273, 
0.333333333333333, 0.307692307692308, 0.357142857142857, 0.333333333333333, 
0.375, 0.411764705882353, 0.388888888888889, 0.421052631578947, 
0.45)), .Names = c("value", "csum", "mean", "run_mean"), row.names = c("1", 
"2", "3", "4", "5", "6", "7", "8", "9", "10", "11", "12", "13", 
"14", "15", "16", "17", "18", "19", "20"), class = "data.frame")
RHertel
  • 23,412
  • 5
  • 38
  • 64