How to compute AUC with ROCR package

Question

I have fitted a SVM model and created the ROC curve with ROCR package. How can I compute the Area Under the Curve (AUC)?

set.seed(1)
tune.out=tune(svm ,Negative~.-Positive, data=trainSparse, kernel ="radial",ranges=list(cost=c(0.1,1,10,100,1000),gamma=c(0.5,1,2,3,4) ))
summary(tune.out)
best=tune.out$best.model

##prediction on the test set
ypred = predict(best,testSparse, type = "class")
table(testSparse$Negative,ypred)

###Roc curve
yhat.opt = predict(best,testSparse,decision.values = TRUE)
fitted.opt = attributes(yhat.opt)$decision.values
rocplot(fitted.opt,testSparse ["Negative"], main = "Test Data")##

Hi, welcome to StackOverflow! You might want to read this: http://stackoverflow.com/questions/5963269/how-to-make-a-great-r-reproducible-example — Tobia Tesan, Jan 09 '17 at 12:54

score 28 · Answer 1 · answered Nov 22 '17 at 15:06

Start with the prediction Method from the ROCR Package.

pred_ROCR <- prediction(df$probabilities, df$target)

to get the ROC in a plot:

roc_ROCR <- performance(pred_ROCR, measure = "tpr", x.measure = "fpr")
plot(roc_ROCR, main = "ROC curve", colorize = T)
abline(a = 0, b = 1)

and get the AUC Value:

  auc_ROCR <- performance(pred_ROCR, measure = "auc")
  auc_ROCR <- auc_ROCR@y.values[[1]]

score 6 · Answer 2 · edited May 23 '17 at 12:01

Your example doesn't seem to be complete, so I can't seem to be able to run it and alter it accordingly, but try plugging in something along the lines of:

...
prediction.obj <- prediction(...)
perf <- performance(prediction.obj, measure = "auc")
print("AUC: ", perf@y.values)

You can append it after sandipan's code, which gives you the plot alone.

Refer to the ROCR manual for performance , page 5: ftp://ftp.auckland.ac.nz/pub/software/CRAN/doc/packages/ROCR.pdf

"auc" is one of the possible measures performance can yield.

Sandipan Dey · Answer 3 · 2017-01-10T12:29:58.640

2

Try this:

tune.out=tune(svm ,Negative~.-Positive, data=trainSparse, kernel ="radial",
              ranges=list(cost=c(0.1,1,10,100,1000),gamma=c(0.5,1,2,3,4), 
              probability = TRUE)) # train svm with probability option true
summary(tune.out)
best=tune.out$best.model
yhat.opt = predict(best,testSparse,probability = TRUE)

# Roc curve
library(ROCR)
# choose the probability column carefully, it may be 
# probabilities[,1] or probabilities[,2], depending on your factor levels 
pred <- prediction(attributes(yhat.opt)$probabilities[,2], testSparse$Negative) 
perf <- performance(pred,"tpr","fpr")
plot(perf,colorize=TRUE)

edited Jan 10 '17 at 12:29

answered Jan 09 '17 at 12:19

Sandipan Dey

21,482
2
51
63

what is df? When I run that command it appear: df$Negative <- as.factor(df$Negative) Error in df$Negative : object of type 'closure' is not subsettable @sandipan – mac gionny Jan 10 '17 at 11:38
df is the original dataframe (which you split into two parts, train and test), you don't need to worry about it, just make it sure that the variable Negative is a factor. – Sandipan Dey Jan 10 '17 at 12:29
a ok because I had made this: tweets$Negative= as.factor(tweets$Sent<=-1) and the other part of your command work well; thank you I will upvote you when I will reach 15 of reputation!! @sandipan – mac gionny Jan 10 '17 at 14:07

score 2 · Answer 4 · edited Aug 13 '18 at 09:47

2

Calculate AUC

# Outcome Flag & Predicted probability
roc_val <-roc(testing.label,gbmPred) 

plot(roc_val,col='blue')

auc(roc_val)

edited Aug 13 '18 at 09:47

Nihal

5,262
7
23
41

answered Aug 13 '18 at 07:07

Saranga

154
1
3

How to compute AUC with ROCR package

4 Answers4

Linked