1

I am using the OptimalCutpoints package in R to find the optimal cutoff point from ROC curve. The criterion for finding the optimal threshold is maximizing Youden's index: 

J = sensitivity + specificity - 1

I am trying to do the same in matlab with the function perfcurve. I run perfcurve with the default criteria for two axis, the FPR in x-coordinates and TPR in y-coordinates. The perfcurve returns a matrix with thresholds and chooses one of them according to the criteria.
The problem is that the optimal threshold that matlab gives, is not the same as in R. However, the optimal threshold according to R is included in the threshold matrix that matlab returns.

How can I replicate the results that R returns with the ones in matlab? I am suspecting that the criteria are not correctly set in matlab for Youden's index.

Schorsch
  • 7,761
  • 6
  • 39
  • 65
Thoth
  • 993
  • 12
  • 36

1 Answers1

1

If you look at the documentation for perfcurve (specifically the OPTROCPT row), you would see that the formula that matlab uses to find the best threshold is quite different, and includes a cost matrix in the optimality criterion.

If you want to replicate what is done in R exactly, use the X and Y return values to compute the Youden index for each threshold, and then choose the best (see how to find max and it's index in array in matlab for some idea how to do it).

Community
  • 1
  • 1
Calimo
  • 7,510
  • 4
  • 39
  • 61
  • Thanks for the reply. I subtract the two matrices (Y-X) and I kept the index of the maximum value as you said. This is the same Youden's index as in R. Also, using the index of the max value, I found the optimal threshold as in R. Thank you! – Thoth Feb 18 '14 at 08:52
  • @Thoth do you need any additional information on this? – Calimo Feb 18 '14 at 10:03
  • No thank you for your interest! My question is answered. I just forgot to mark your post as the answer. – Thoth Feb 19 '14 at 20:33