Automation of a calculation in a nested list

Question

I’m working with a large list that contains measured values of 134 different sensors. Each sensor has measurement values for different years (e.g. sensor 1 has values for 2014, 2015 and 2016 | sensor 2 has values for 2016, 2017 and 2018). You can see the data structure here:

List of 134
 $ SE3  :List of 3
  ..$ 2016:'data.frame':    366 obs. of  1 variable:
  .. ..$ SE3_: num [1:366] 68.4 68.4 68.4 68.4 68.4 ...
  ..$ 2017:'data.frame':    365 obs. of  1 variable:
  .. ..$ SE3_: num [1:365] 68.7 68.6 68.6 68.6 68.6 ...
  ..$ 2018:'data.frame':    365 obs. of  1 variable:
  .. ..$ SE3_: num [1:365] 68.4 68.4 68.3 68.3 68.3 ...
 $ SE4  :List of 2
  ..$ 2017:'data.frame':    365 obs. of  1 variable:
  .. ..$ SE4_: num [1:365] 40.4 40.4 40.3 40.2 40.3 ...
  ..$ 2018:'data.frame':    365 obs. of  1 variable:
  .. ..$ SE4_: num [1:365] 44.9 46.6 46.7 45.7 45.4 ...
 $ SE6  :List of 5
  ..$ 2014:'data.frame':    365 obs. of  1 variable:
  .. ..$ SE6_: num [1:365] 29.2 29.1 29.2 29.3 29.4 ...
  ..$ 2015:'data.frame':    365 obs. of  1 variable:
  .. ..$ SE6_: num [1:365] 29.3 29.4 30.2 30.1 29.7 ...
  ..$ 2016:'data.frame':    366 obs. of  1 variable:
  .. ..$ SE6_: num [1:366] 28.8 28.8 28.8 29.1 29.3 ...
  ..$ 2017:'data.frame':    365 obs. of  1 variable:
  .. ..$ SE6_: num [1:365] 29.2 29.1 29.1 29.1 29.2 ...
  ..$ 2018:'data.frame':    365 obs. of  1 variable:
  .. ..$ SE6_: num [1:365] 30.1 30.4 30.7 30.3 30.2 ...

Here is what I want to do (as an example for SE3 in the list, but I want to do it for every sensor that is in the list):

1. I want to calculate the maximum out of all (in this example three) dataframes. So for example if the maximum of year 2016, 2017 and 2018 is 76, 69 and 63 then I want to use 76 as it is the maximum out of all dataframes.
2. I want to use that maximum value as variable ‘Smax’ in the following formula: B = P-(P*Smax/W) with P = 2.65 and W = 100. I wrote a function for that:

    calculate_B <- function(P, Smax, W) { 

          B = P-(P*Smax/W)

    } 

3. Then I wanna use the calculated ‘B’ in another formula for each row of a dataframe which is: W = S/1-(B/P)) where the variable ‘S’ in the formula is each measurement value that is stored in the list (so it is not Smax anymore), P is 2.65 again and B is the value that I calculated in step 2. I already wrote a function that calculates ‘W’ which is:

       calculate_WFPS <- function(S, B, P) { 

        S/ (1 – (B/P)) 

        } 

I do know how to do some of these steps manually, but I can’t automate them as I lack knowledge.

These are my ideas: For the first step I can use sapply as follows:

sapply(list_sensors_d20, function(x) 
  sapply(x, function(y)
    sapply(y, max)))

But then there’s the problem that the vector with the maximum values contains 3 maximums for SE3 even though I only want to have one maximum and then use that as ‘S’ in step 2. For the third step (for one dataframe) I could use:

apply(data, 1, fun_WFPS, B = CALCULATED VALUE FROM STEP 2, P = 2.65) 

But again, I don’t know how to automate these steps. Can anyone help me out with this? Please don’t hesitate to ask any questions if something is not clear.

I thought it might be a good idea to make a full example of the calculation (only for sensor 3) so here it is (numbers 1, 2 and 3 refer to the steps I explained earlier):

#step 1
temp_max <- sapply(list_sensors_d20$SE3, function(x)
  sapply(x, max))
Smax <- max(temp_max) 
#step 2 (using the function ‘calculate_B’) 
B_for_step3 <- calculateB(P = 2.65, Smax = Smax, W = 100)
#step3 (using the function ‘calculate_WFPS’) 
New_df <- data.frame(apply(list_sensors_d20$SE3$`2016`, 1, fun_WFPS, B = B_for_step3, P = 2.65)) 

Thanks in advance, I’d really appreciate some help!

Answer 1

I think you may be over complicating parts of this.

Example Data:

list_sensors_d20 <- list(SE3 = list('2016'=c(1:6,2:7,3:8),'2017'=c(1:6,2:7,3:8)),SE4 = list('2016'=c(21:26,2:7,3:8),'2017'=c(1:6,2:7,3:8)))

Functions (Note: I rearanged the order of the variables so that they can be passed into sapply more easily):

calculate_B <- function(Smax, P, W) {
    B = P-(P*Smax/W)} 

calculate_WFPS <- function(S, B, P){
    S/(1-(B/P))}

Code:

Smax <- sapply(list_sensors_d20, function(x) max(unlist(x)))
B <- sapply(Smax, calculate_B, P=2.65, W=100)
WFPS <- lapply(1:length(B), function(x) sapply(unlist(list_sensors_d20[x],recursive = F), calculate_WFPS, B[x], 2.65)) 

Outputs:

> Smax 
SE3 SE4 
  8  26 

> B 
  SE3   SE4 
2.438 1.961 

> WFPS 
[[1]]
      SE3.2016 SE3.2017
 [1,]     12.5     12.5
 [2,]     25.0     25.0
 [3,]     37.5     37.5
 [4,]     50.0     50.0
 [5,]     62.5     62.5
 [6,]     75.0     75.0
 [7,]     25.0     25.0
 [8,]     37.5     37.5
 [9,]     50.0     50.0
[10,]     62.5     62.5
[11,]     75.0     75.0
[12,]     87.5     87.5
[13,]     37.5     37.5
[14,]     50.0     50.0
[15,]     62.5     62.5
[16,]     75.0     75.0
[17,]     87.5     87.5
[18,]    100.0    100.0

[[2]]
        SE4.2016  SE4.2017
 [1,]  80.769231  3.846154
 [2,]  84.615385  7.692308
 [3,]  88.461538 11.538462
 [4,]  92.307692 15.384615
 [5,]  96.153846 19.230769
 [6,] 100.000000 23.076923
 [7,]   7.692308  7.692308
 [8,]  11.538462 11.538462
 [9,]  15.384615 15.384615
[10,]  19.230769 19.230769
[11,]  23.076923 23.076923
[12,]  26.923077 26.923077
[13,]  11.538462 11.538462
[14,]  15.384615 15.384615
[15,]  19.230769 19.230769
[16,]  23.076923 23.076923
[17,]  26.923077 26.923077
[18,]  30.769231 30.769231