0

I have to build a couple of models and validate their stability through 10 fold CV

I have the following code for a logistic regression and nnet for example 

mod_fitlog <- train(form,  data=x, method="glm", family="binomial",
                 trControl = ctrl, tuneLength = 5,verbose=T,metric='auc')

mod_fitnnet <- train(form,  data=x, method="nnet",preProcess = "range",
                trace = FALSE, maxit = 100)

Now i want to extract the results of 10 cv runs , i.e. 10 aucs for each modelling and plot them - basically to judge which model is more stable and which isnt with higher variance .

jmuhlenkamp
  • 2,102
  • 1
  • 14
  • 37
Learner_seeker
  • 544
  • 1
  • 4
  • 21
  • It's easier to help if you provide a [reproducible example](http://stackoverflow.com/questions/5963269/how-to-make-a-great-r-reproducible-example) with sample input data so we can run the code. – MrFlick Mar 23 '17 at 02:43

1 Answers1

0

I'd suggest you look at pipelearner which was written for exactly this kind of task.

neilfws
  • 32,751
  • 5
  • 50
  • 63