1

I'm running a support vector machine with a radial basis kernel function in the R caret package. My code runs without errors or warnings, however it generates a "maximum number of iterations reached ..." message which I interpret as meaning the algorithm didn't converge to a solution.

Using a small college admissions dataset (4 features, n=400) as an example:

# Load data & factor admit variable.
> mydata <- read.csv("http://www.ats.ucla.edu/stat/data/binary.csv")
  mydata$admit <- as.factor(mydata$admit)

# Create levels yes/no to make sure the the classprobs get a correct name.
 levels(mydata$admit) = c("yes", "no")


# Train SVM via 10-fold CV.
set.seed(123)
train_control <- trainControl( method="cv",
    number=10,
    classProbs = TRUE,
    savePredictions = TRUE)

model_rbfsvm<- train(as.factor(admit) ~ .,
    data=mydata,
    trControl=train_control,
    method="svmRadial", 
    family="binomial", 
    tuneGrid=expand.grid(C=c(.000001, .00001, .0001, .001, .01, .1, 1, 10), sigma=c(.00001, .0001, .001, .01, .1, 1, 10)), 
    metric="Accuracy", 
    preProcess=c("center","scale"))

successfully executes but produces the following message (I've abbreviated - it goes on for many lines):

maximum number of iterations reached 4.663775e-05 4.663771e-05maximum number of iterations reached 0.0003396917 0.0003396878...

Adjusting the maximum number of iterations with the maxiter= statement in the train function produced the same message.

Is it possible to adjust the maximum number of iterations in caret or is it fixed at a specific level?

RobertF
  • 824
  • 2
  • 14
  • 40

1 Answers1

1

It is not possible to adjust the number of iterations. If you specify maxiter = ....., it is passed on to the underlying kernlab lssvm function, but is not an option in the lssvm function and is ignored.

Your warning only happens when you set classProbs = TRUE, if you leave it on the default option, you will not see a message. Which is interesting because the message indicates that svm hasn't reached convergence yet and somehow it does when classProbs = FALSE. But I have my doubts about it, because the model outcome is exactly the same. If you set verboseIter = TRUE the messages are a bit better displayed. 

+ Fold1: C=0.001, sigma=0.001 
maximum number of iterations reached 0.00024476 0.0002447579- Fold1: C=0.001, sigma=0.001 
+ Fold1: C=0.010, sigma=0.001 
maximum number of iterations reached 0.002770727 0.002765972- Fold1: C=0.010, sigma=0.001

Running the model directly with the lssvm function doesn't return any message with maximum number of iterations. You might want to drop a line on the github page of caret.

P.S.: the family option you specify in the train function is not needed (and not used).

phiver
  • 23,048
  • 14
  • 44
  • 56
  • Thanks @phiver, this is very helpful. Yea, I don't see any `maxiter=` arguments in the `lssvm` function, so I'm assuming the algorithm only terminates when the tolerance criterion is met (default = .0001). Despite the "maximum number of iterations reached ..." messages, `caret` is still outputting SVM predictions so this must be a spurious warning. – RobertF May 04 '16 at 15:44
  • 1
    After a little more digging, I came across this post: http://stackoverflow.com/questions/15503027/why-are-probabilities-and-response-in-ksvm-in-r-not-consistent], which discusses separate methodologies for fitting the SVM when `classProbs = TRUE` (where predicted response is a probability) vs. `classProbs = FALSE` (where predicted response is a yes/no label). Possibly we see the maximum iterations message when `classProbs = TRUE` because the algorithm isn't converging. – RobertF May 04 '16 at 20:00