Calculate statistical parameters (quantile and median) from a serie of ECDF with R

Question

Edit:

Okay, sorry i will try to be more clear,

I have 50 scenarios (here i create it randomly), and i put it all of this scenarios in a matrix. After i can apply the ecdf function, that give me a list of 50 ecdf. And i want to calculate, from all this ecdf of my 50 scenarios, the both quantiles 90 and 10 and the median.

This is a basic code:

ma <- matrix(ncol = 50, nrow = 200)
for (i in 1:50) {
     x <- runif(1:200, min = 0, max = 100)
     ma[,i] <- x
}
ma_ecdf <- apply(ma, 2, ecdf)


plot(ma_ecdf[[1]])
for (i in 1:50) {
     lines(ma_ecdf[[i]])
}

So i can plot all of them easily, but i just want to represent the three parameters (Q10, Q50, Q90) on a graph.

Edit:

I found exactly how do it, so i share it, if sometimes someone need it.

you can try the code, the graphic is very explicit, and explains well what i wanted to do. Thx for people which tried to help me!

ma_data <- matrix(ncol = 50, nrow = 200)

for (i in 1:50) {
     a <- runif(1:200, min = 0, max = 100)
     ma_data[,i] <- a
}

ma_ecdf <- apply(ma_data, 2, ecdf)
x <- seq(from = 0, to = 1, by =0.1)
ma <- matrix(ncol = 50, nrow = length(x))

for (i in 1:length(x)) {    
     prob <- x[i]
     for (j in 1:length(ma_ecdf)){
     ma[i,j] <- quantile(ma_ecdf[[j]], probs = prob)
     }
}

q10 <- apply(ma, 1, quantile, probs = c(0.10))
q90 <- apply(ma, 1, quantile, probs = c(0.90))
med <- apply(ma, 1, median)

plot(ma_ecdf[[1]])
for (i in 2:50) {
     lines(ma_ecdf[[i]])
}
lines(med, x, type = 'o', col = 'red', lwd = 2)
lines(q90, x, type = 'o', col = 'green', lwd = 2)
lines(q10, x, type = 'o', col = 'green', lwd = 2)

You can choose to plot all the ecdf with the both quantile and median, or just the quantiles and median to make it more clear.

Can you show as an MWE (Minimal Working Example) of the code that you already have? That would help us to guide you further. — ph0t0nix, May 04 '15 at 08:35
Please read about providing a [reproducible example](http://stackoverflow.com/questions/5963269/how-to-make-a-great-r-reproducible-example). — r2evans, May 04 '15 at 08:37
Shouldn't be that difficult to make a list of `ecdf`'s and then use `lapply` or `sapply` to loop through them with a function that returns the desired quantiles. It's unclear what it means to say "for each lines of the y axis" — IRTFM, May 04 '15 at 09:09
Your question is very unclear. Couldn't you simply calculate the qunatiles from the raw data? `apply(ma, 2, quantile, prob = c(0.1, 0.5, 0.9))` — Roland, May 04 '15 at 09:09
I don't want to calculate directly the probability on my data, i need to calculate it from the ecdf function. So from each density value. So on each density value (y axis) i have 50 values, one from each ecdf, and i want to take the Q10, Q50 and Q90. And if i used `lapply` or `sapply`, like this: `test <- lapply(ma_ecdf, median)` , i have a message error: `object of type 'closure' ` — User123456789, May 04 '15 at 09:24
The ECDF is a function (as implied by the fourth letter of the acronym) so trying to take the median of a function creates an error. You need to recover the data that was used in building the function. The `ecdf` function stores that data in the environment attached to the function. — IRTFM, May 04 '15 at 09:30

IRTFM · Answer 1 · 2015-05-04T09:49:33.000

The efun-function extracts the environment of the ecdf's and then calculates the desired quantiles. (It's simply incorrect to suggest that you cannot pass a list of functions to members of the *apply family although I suppose apply might do violence to a lsit of functions. But why you would want to use apply is not articulated.)

The plotting request is unclear, so I'm just supplying a matplot. (I think the interesting part is done.)

 efun <- function(fn) { e <- environment(fn); quantile( e$x, prob=c(0.1, 0.5, 0.9) ) }
 sapply(ma_ecdf, efun)
 #------------
        [,1]      [,2]     [,3]      [,4]     [,5]     [,6]     [,7]     [,8]      [,9]
10% 11.20981  9.694139 11.07211  8.129253 10.18672 10.20660 13.42461 10.09662  9.155876
50% 50.86365 45.399646 50.09159 52.472317 45.83210 47.82140 52.37679 45.30424 51.370869
90% 90.73561 90.308808 87.72453 90.409360 89.66196 88.30103 92.96570 87.55434 91.887313
       [,10]    [,11]    [,12]   [,13]    [,14]     [,15]    [,16]    [,17]     [,18]
10% 11.52653 10.13625 10.04209 10.2433 10.14813  8.114616 13.86385 11.24537  8.682568
50% 49.29966 50.59389 52.60945 45.3897 51.34410 46.912610 54.52146 50.86271 49.204883
90% 87.62950 92.15806 91.54697 89.8588 92.53752 91.157058 86.73797 93.53906 90.686209
       [,19]    [,20]    [,21]     [,22]    [,23]    [,24]     [,25]    [,26]    [,27]
10% 13.42698  9.83722 12.50920  9.042764 12.68967 10.81326  7.331495 10.97554 12.82455
50% 53.85215 53.03308 53.53052 46.258026 53.21290 47.76353 40.680560 47.83468 48.76479
90% 88.38150 87.50191 89.57422 93.140304 91.31335 92.64003 87.679489 86.44366 87.89013
        [,28]    [,29]     [,30]     [,31]     [,32]    [,33]    [,34]     [,35]
10%  9.478504 11.97249  9.288765  8.023545  9.167379 11.97052 10.81782  9.129501
50% 51.256558 50.08606 42.848092 49.300343 51.131813 51.21670 43.35010 47.818362
90% 91.601705 86.56648 84.462400 91.899195 86.919949 90.47939 90.89439 89.810636
       [,36]     [,37]    [,38]    [,39]     [,40]    [,41]    [,42]    [,43]     [,44]
10% 16.61909  9.579045 10.96399 14.04819  8.941116 11.42047 11.16979 10.74832  8.836482
50% 57.78916 49.060583 52.84561 54.79853 48.950509 56.18923 46.80874 46.82841 50.649137
90% 92.04979 91.504328 90.61166 91.75224 89.978594 91.10922 88.41800 86.04107 92.654152
       [,45]    [,46]     [,47]    [,48]     [,49]    [,50]
10% 12.34652 10.10571  7.374205 11.54974  7.079834 11.04556
50% 48.43045 56.15222 47.624488 45.90533 52.843572 51.11976
90% 88.18035 91.85914 91.297291 89.62237 92.382659 91.92695

enter image description here

If it is desired to link all of the quantile_0.10 values together (and hte medians and 90th percentiles using the sequence of the function as the x-coordinate, then this plot is the result:

png();matplot( 1:50, t(sapply(ma_ecdf, efun)) , type="b"); dev.off()

enter image description here

The result is identical to `apply(ma, 2, quantile, prob = c(0.1, 0.5, 0.9))`. — Roland, May 04 '15 at 09:27
That's good. (The requested task was to do it with the ECDF's.) — IRTFM, May 04 '15 at 09:29
But I think, they want to actually reduce their 50 ecdfs to 3 lines with the "quantiles" from these lines. (Although this doesn't make sense from the statistics.) I.e., for each x value they seem to want to extract the corresponding probabilities and calculate the quantiles from these. — Roland, May 04 '15 at 09:30
This is not exactly what i want, i want to, at the end, juste have three curves, one for the Q10, one for the Q50 and one for the Q90. Not the quantiles of each one scenarios, but of all the 50 on each density values. — User123456789, May 04 '15 at 09:38
Unfortunately I have found this description mostly incomprehensible. I'll post another guess at what it might mean. — IRTFM, May 04 '15 at 09:47
At the end, I want to reduces the 50 ecdfs into three ecdfs each representing the 'Quantile' Q10, Q50 and Q90 of the 50 ecdfs scenarios. In each y value, you have 50 points, from the 50 ecdfs, which can be represented as a normal distribution of x values. And i want the 'quantiles' of this values. — User123456789, May 04 '15 at 12:12

Calculate statistical parameters (quantile and median) from a serie of ECDF with R

1 Answers1