3

I am trying to classify the yard digits on the football field. I am able to detect them (different method) well. I have a minimal bounding box drawn around the tens place digits '1,2,3,4,5'. My goal is to classify them.

Ive been trying to train an SVM classifier on hog features I extract from the training set. A small subset of my training digits are here: http://ssadanand.imgur.com/all/

While training, I visualize my hog descriptors and they look correct. I use a 64X128 training window and other default parameters that OPencv's HOGDescriptor uses.

Once I train my images (50 samples per class, 5 classes), I have a 250X3780 training vector and 1X250 label vector which holds the class label values which I feed to a CvSVM object. Here is where I have a problem.

I tried using the default CvSVMParams() while using CvSVM. Terrible performance when tested on the training set itself!

I tried customizing my CvSVMPARAMS doing this:

CvSVMParams params = CvSVMParams();
params.svm_type = CvSVM::EPS_SVR;
params.kernel_type = CvSVM::POLY;
params.C = 1; params.p = 0.5; params.degree = 1;

and different variations of these parameters and my SVM classifier is terribly even when I test on the training set!

Can somebody help me out with parameterizing my SVM for this 5 class classifier? I don't understand which kernel and what svm type I must use for this problem. Also, how in the world am I supposed to find out the values of c, p, degree for my svm?

I would assume this is an extremely easy classification problem since all my objects are nicely bounded in a box, fairly good resolution, and the classes i.e.: the digits 1,2,3,4,5 are fairly unique in appearance. I don't understand why my SVM is doing so poorly. What am I missing here?

Vikram
  • 3,996
  • 8
  • 37
  • 58
Sreemanananth Sadanand
  • 233
  • 1
  • 3
  • 7

1 Answers1

4

A priori and without experimentation, it's very hard to give you some good parameters but I can give you some ideas.

First, you want to model a multi class classifier but you are using a regression algorithm, not that you can't do that but usually is easier if you start with C-SVM first.

Second, I would recommend to use RBF instead of a Polynomial kernel. Poly is very hard to get it right and usually RBF would do a better job out of the box.

Third, I would play with several values of C, don't be shy and try a bigger C (such as 100) which would force the algorithm to pick more SVs. It can lead to overfitting but if you can't even make the algorithm to learn the training set that's not your immediate problem.

Fourth, I would reduce the dimension of the images at first and then if needed, when you have a more stable model, you could try with that dimension again.

I really recommend you to read LibSVM guide which is very easy to follow http://www.csie.ntu.edu.tw/~cjlin/papers/guide/guide.pdf

Hope it helps!

EDIT:

I forgot to mention, that a good way to pick parameters for SVM is to perform cross-validation: http://en.wikipedia.org/wiki/Cross-validation_(statistics)

http://www.autonlab.org/tutorials/overfit10.pdf

http://www.youtube.com/watch?v=hihuMBCuSlU

http://www.youtube.com/watch?v=m5StqDv-YlM

EDIT2:

I know is silly because it's on the title of the question, but I didn't realize you were using HOG descriptors until you pointed out on the comments.

Pedrom
  • 3,823
  • 23
  • 26
  • Thank Pedrom, Much appreciated. I did try using PCA on my training data to reduce my dimensionality but heres the catch, i have a 3780 feature vector and only 50 instances per class. I think that counts as very little data to identify principal components from. Even to retain 10% of my datas variance, I need to use all 50 eigen vectors. I can produce more data for training, but Im unsure if it'll help. Can you give me an intuitive explanation of the SVM parameters??? – Sreemanananth Sadanand Mar 20 '13 at 20:20
  • Like for instance you just mentioned, higher the c value, more the sv's picked up, how would I intuitively understand what the rest of the parameters do? – Sreemanananth Sadanand Mar 20 '13 at 20:23
  • @SreemanananthSadanand Well PCA is nice to reduce the dimension of your input but I was thinking in something more simple like resize the images :) As for how intuitively select the parameters well it would depend of the parameter but let's talk about the C and gamma (the parameter asked on RBF). C is the penalty cost for violating the linear separability, so for a lower C the algorithm doesn't have to pay much for those unclassified points so it can be more permissive and select fewer SVs (I am oversimplifying here but the analogy works). – Pedrom Mar 20 '13 at 20:42
  • @SreemanananthSadanand So with a higher C, the algorithm will need to pay more for those not classified points, so it will try harder to have as few unclassified points as possible (Not entirely true but again is just an intuitive way to see it). Short story, higher C as closer to overfitting and lower C to missclassification. This question might help you further on this http://stackoverflow.com/questions/9480605/what-is-the-relation-between-the-number-of-support-vectors-and-training-data-and – Pedrom Mar 20 '13 at 20:45
  • @SreemanananthSadanand The gamma parameter of the RBF kernel has another analogy that might help you to visualize what is going on. You can think RBF as hyperspheres in feature space that will be "activated" if a point is close enough to the center. So you can think that gamma as the inverse of the radius of that hypersphere. So for bigger values of gamma it would be an small hypersphere and for lower gammas the hyphersphere will be bigger. These images might help to see it better http://goo.gl/S4EZx http://goo.gl/A1jHy Let me know if I can help you further – Pedrom Mar 20 '13 at 20:55
  • Brilliant visuals! Very helpful. Let me chew on this, I know HOG descriptors will do well in this problem, its just a matter of finding the right SVM parameters to train. Thank you sir! – Sreemanananth Sadanand Mar 21 '13 at 13:50