1

I am trying to calculate the cut-off point that max sensitivity vs specifity. I am using the ROCR package and I have managed to plot the graph sensitivity vs specifity. However, I don't know how to calculate what is the cut off point that max sensitivity vs specifity. Ideal I would like to have a label in the graph that shows the cut off and the coordenates at the point. But, any suggestion to solve this question will be greatly appreciated.

pred <- prediction( ROCR.simple$hello, ROCR.simple$labels)
ss <- performance(pred, "sens", "spec")
plot(ss)
Rita A. Singer
  • 167
  • 3
  • 7
  • can you elaborate on what you mean by the "cut-off point that max sensitivity vs specificity"... in a ROC curve, max sensitivity=max specificity = 1.0 – JHowIX Aug 23 '15 at 21:48
  • Did you run the code in your example? AFAICT, there is no `hello` element in `ROCR.simple`. – jlhoward Aug 23 '15 at 21:51
  • @JHowIX I think you are talking about sensitivity vs. 1-specificity (aka true positive rate vs. false positive rate). If you achieve sensitivity and specificity of 1 then you have a perfect prediction model. – josliber Aug 23 '15 at 22:02

1 Answers1

3

"Maximize sensitivity vs. specificity" isn't very precise, because you are trading these quantities off at each point along the ROC curve. To make it more precise, I'll assume you are trying to maximize the sum of these two values. Let's look at your example, which uses ROCR.simple:

library(ROCR)
data(ROCR.simple)
pred <- prediction(ROCR.simple$predictions, ROCR.simple$labels)
ss <- performance(pred, "sens", "spec")
plot(ss)

enter image description here

You can identify the cutoff that yields the highest sensitivity plus specificity with:

ss@alpha.values[[1]][which.max(ss@x.values[[1]]+ss@y.values[[1]])]
# [1] 0.5014893
max(ss@x.values[[1]]+ss@y.values[[1]])
# [1] 1.69993

The highest sensitivity plus specificity is achieved in this case when you predict the positive outcome when the predicted probability exceeds 0.501 and predict the negative outcome when the predicted probability does not exceed 0.501. This yield a sensitivity plus specificity value of 1.7.

Naturally, this can be extended to other functions of the sensitivity and specificity by changing the expression inside the which.max call.

josliber
  • 43,891
  • 12
  • 98
  • 133
  • Many thank Josilber. I think the better way to visualize this would be a plot pf sensitivity vs specificity at various cutoff points( x-axis) where the intersection of sensitivity vs specificity is at the optimal cut-point. Any suggestion of how to plot this.But I am not sure how to plot this.Another option could be to highligh in the graph the coordenate that maximizes sensitivity + specificity. – Rita A. Singer Aug 29 '15 at 18:17
  • 1
    @RitaA.Singer you can access the specificity with `ss@x.values[[1]]`, the sensitivity with `ss@y.values[[1]]`, and the cutoff with `ss@alpha.values[[1]]`, so making such a plot (or any other variant) would be simple. – josliber Aug 29 '15 at 22:45
  • Thank Josilber. I was a bit confused before.I have made work now. – Rita A. Singer Aug 30 '15 at 11:23