How to use OneVsRestClassifier with SVC for multilabel problems?

Question

I'm using OneVsRestClassifier for multilabel classification. It works with LinearSVC, but when I apply it to SVC, the following error appears:

classifier = OneVsRestClassifier(SVC(class_weight='balanced'))
classifier.fit(X1, y1)
y2 = classifier.predict(X2)

Traceback (most recent call last):
...
File "/usr/local/lib/python2.7/dist-packages/sklearn/multiclass.py", line 219, in predict
  return predict_ovr(self.estimators_, self.label_binarizer_, X)
File "/usr/local/lib/python2.7/dist-packages/sklearn/multiclass.py", line 93, in predict_ovr
  Y = np.array([_predict_binary(e, X) for e in estimators])
File "/usr/local/lib/python2.7/dist-packages/sklearn/multiclass.py", line 66, in _predict_binary
  score = estimator.predict_proba(X)[:, 1]
File "/usr/local/lib/python2.7/dist-packages/sklearn/svm/base.py", line 490, in predict_proba
  "probability estimates must be enabled to use this method")
NotImplementedError: probability estimates must be enabled to use this method</code>

Does anybody know what is it?

t looks like you are trying to obtain probabilistic output but you forgot to include probability estimates when training the models. Can you post the training commands? — Marc Claesen, May 06 '13 at 15:47
It's very straightforward: `classifier = OneVsRestClassifier(SVC(class_weight='auto')); classifier.fit(X1, y1); y2 = classifier.predict(X2)` The error occurs while predicting (in the last line). The same code works for LinearSVC, but does not for SVC, so that surprises me. — lizarisk, May 07 '13 at 07:22

score 5 · Accepted Answer · edited May 23 '17 at 12:22

5

This is a bug. The OneVsRestClassifier calls the predict_proba method when it finds one, but the one on SVC does not actually work unless you construct it with probability=True to get Platt scaling (which I don't actually encourage).

The reason that it works for LinearSVC is that that class does not have a predict_proba, so OvR backs off to the decision_function method.

edited May 23 '17 at 12:22

Community

1
1

answered May 07 '13 at 14:11

Fred Foo

355,277
75
744
836

Has this been fixed yet? If not is there a temporary fix to get this to work? – willwest Jun 21 '14 at 22:29
I'm still seeing the issue in version 0.14.1. My code is identical to OP's except I'm using a Pipeline with SelectKBest preceding the SVC(). – willwest Jun 22 '14 at 20:07
1

@willwest Strange. But in any case, it works fine in 0.15b1, the current beta. – Fred Foo Jun 23 '14 at 09:17

How to use OneVsRestClassifier with SVC for multilabel problems?

1 Answers1