Predicting Class in Multiclass classification using SVM

Question

In:svc1.predict([X_poly[25].reshape(1,-1))

Out:array([3], dtype=int64)

In:svc1.decision_function([X_poly[25]])

Out:array([[ 0.58348329, -0.99979492,  0.08571431, -0.99996706, -0.52397444,
     0.99959056]])

In:svc1.predict_proba(X_poly[25].reshape(1,-1))

Out:array([[ 0.48373954,  0.11870606,  0.00261101,  0.39494339]])

I am using rbf kernel and 'ovo' classification. I am trying to predict the class of 26th sample in the data. The SVC classifier predicted the class sample as 3. However, I am not able to comprehend how does it narrow down to class 3 when probabilities suggest the other way. Can you also please advise the interpretation of the output of decision_function method and how it is used to predict the class.

Possible duplicate of [Scikit-learn predict\_proba gives wrong answers](https://stackoverflow.com/questions/17017882/scikit-learn-predict-proba-gives-wrong-answers) — Vivek Kumar, Jan 22 '18 at 09:49
Also see this question: https://stackoverflow.com/questions/15111408/how-does-sklearn-svm-svcs-function-predict-proba-work-internally — Vivek Kumar, Jan 22 '18 at 09:49

score 1 · Accepted Answer · answered Jan 22 '18 at 09:17

As the documentation suggests:

The probability model is created using cross validation, so the results can be slightly different than those obtained by predict. Also, it will produce meaningless results on very small datasets.

I think this is why you have different outputs.

As for the ovo (one-vs-one) decision function, since your decision function returns 6 values that means you have 4 classes:

(n*(n-1))/2 = 6, where n is the number of classes.

Now for how to predict the class using the numbers returned from the decision function. As the name suggests (one-vs-one), it compares the classes one-to-one and then outputs a positive or a negative number on which class it thinks is the right one. In your case, and since we have 4 classes, it produces 6 numbers which represent the comparison between your classes (1,2,3,4) as follows: [12,13,14,23,24,34] which is represented in your case by this decision vector:

[0.58348329, -0.99979492,  0.08571431, -0.99996706, -0.52397444, 0.99959056]

So, which class wins depends on the signal of the numbers, so in this example, the winner classes are: [1,3,1,3,4,3], i.e. '12' is compared by 0.58348329, hence class '1' wins ... etc. Therefore, the decision function votes the class '3' as your prediction since it is the most winning class.

Predicting Class in Multiclass classification using SVM

1 Answers1