5

I use liblinear with my program to perform multi-class classification with the L2R_L2LOSS_SVC_DUAL solver. In the current test-setup I have 1600 instances from a total of 9 classes with 1000 features each.

I'm trying to determine the optimal C parameter for training with 5-fold cross-validation, but even with a small C of 1.0 liblinear reaches the maximal number of iterations:

................................................................................
....................
optimization finished, #iter = 1000

WARNING: reaching max number of iterations
Using -s 2 may be faster (also see FAQ)

Objective value = -637.100923
nSV = 783

The FAQ site mentions two possible reasons for this:

  1. Data isn't scaled.
  2. A large C parameter is used.
  3. A lot of instances with a small number of features is used, so that the solver L2R_L2LOSS_SVC may be faster.

Neither one applies to my case. Since my feature vector is some kind of histogram, there is a natural maximum, that I use to scale the features to [0,1].

I set up the parameteres for liblinear as follows:

struct parameter svmParams;
svmParams.solver_type = L2R_L2LOSS_SVC_DUAL;
svmParams.eps = 0.1;
svmParams.nr_weight = 0;
svmParams.weight_label = NULL;
svmParams.weight = NULL;
svmParams.p = 0.1;
svmParams.C = 1.0;

My question is: What other reasons, not mentioned in the FAQ, may cause liblinear to operate slow in this scenario and what may I do against it?

Callidior
  • 2,899
  • 2
  • 18
  • 28
  • I have the same problem in liblinear,and I use -s 2 to solve this. I'm interesting in this question. – mickeyandkaka Aug 07 '14 at 10:44
  • @mickeyandkaka: I have tried -s 2 too, but in my case it degrades accuracy. – Callidior Aug 07 '14 at 10:48
  • 1000 iterations of SVM solver is very low, why would you set such limitation? Use at least 1,000,000 and tell us whether you still get iterations limit reached error – lejlot Aug 07 '14 at 17:48
  • @lejlot That limitation is hard-coded in liblinear. They probably have some reason for that. Beyond that, doing all the 1000 iterations already is way too slow for my purpose of parameter optimization by cross-validation. – Callidior Aug 07 '14 at 18:32
  • http://stats.stackexchange.com/questions/37669/libsvm-reaching-max-number-of-iterations-warning-and-cross-validation – Issam Zoli Aug 13 '14 at 06:28
  • Looked into liblinear source code and the library looks pretty unreliable to me. One of the reasons is that they use, `new`, `malloc` and `realloc` inconsistently and use the pointer without checking if the allocation call failed. For people who could help OP better: `solve_l2r_l1l2_svc(prob, w, eps, Cp, Cn, L2R_L2LOSS_SVC_DUAL);` in `linear.cpp` in the source tar – askmish Aug 16 '14 at 06:46

0 Answers0