0

I am trying to plot a ROC curve for my glmnet regression model. In order to do that, I am trying to predict using type = "response" in predict function: 

 pred_glmnet_s10_2class <- predict(model_train_glmnet_s10_2class,
                                   newdata=testing_s10_2class,
                                   s = "model_train_glmnet_s10_2class$finalModel$lambdaOpt",
                                   type="response")

and I get the following error:

Error in predict.train(model_train_glmnet_s10_2class, newdata = testing_s10_2class, : type must be either "raw" or "prob"

My predictions and class labels are binary 0 and 1 and have been factored. Any help is really appreciated. Also, any ideas on how to plot AUC (Area Under ROC curve) vs number of features? Thanks!

steveb
  • 5,382
  • 2
  • 27
  • 36
trickymaverick
  • 199
  • 1
  • 3
  • 8
  • 1
    Can you provide a [minimal reproducible example](http://stackoverflow.com/questions/5963269/how-to-make-a-great-r-reproducible-example)? – shayaa Aug 15 '16 at 05:32
  • For your follow up questions on AUC, what have you tried ? – steveb Aug 15 '16 at 05:40
  • `predict` is generic. You need to know the class of `model_train_glmnet_s10_2class`. Then you can determine which of the many `predict` functions you are actually using. It's possible that there is no `type="response"` to that class of object. – IRTFM Aug 15 '16 at 06:17
  • The error message says it all! `Type` needs to be either `raw`or `prob`. `raw` is the default and returns the classification response (1 or 0). prob returns the probabilities. – phiver Aug 15 '16 at 06:51
  • @42- the error message answers the question: `predict.train`. – Calimo Aug 15 '16 at 07:25
  • From the error message it would appear that "response" is not valid string for the type parameter. Indeed this is confirmed from the Caret [package documentation](https://cran.r-project.org/web/packages/caret/caret.pdf) on CRAN. – davidrpugh Aug 15 '16 at 08:09
  • @Calimo; yes, I know. It's an enduring puzzle to my why people don't read error messages or help pages. – IRTFM Aug 15 '16 at 20:08

1 Answers1

2

Assuming that model_train_glmnet_s10_2class was generated by train (showing the code would be helpful)...

Using predict(model_train_glmnet_s10_2class) is using predict.train and uses the optimal lambda values determined by train automatically. If you want the probabilities, just use type = "prob".

Your syntax is consistent with predict.glmnet and not predict.train.

As said in the documentation, it is a really bad idea to use model_train_glmnet_s10_2class$finalModel directly to do the predictions

topepo
  • 13,534
  • 3
  • 39
  • 52