I'm using a LinearSVC-model for making a prediction between 5 classes and using CalibratedClassifierCV-model to get "probabilities" for each class. Here are my definition of the models:
from sklearn.calibration import CalibratedClassifierCV
from sklearn.svm import LinearSVC
model = LinearSVC()
clf = CalibratedClassifierCV(model,cv="prefit", method="sigmoid")
model.fit(X_train, y_train)
clf.fit(X_train, y_train)
Theoretically my guess would be that "model" and "clf" would make similiar predictions. For most cases (maybe 98%) they are, but for some cases (which I can't give reproducible code for), they don't, so my question is just theoretical.
When I run:
print(model.classes_)
print(clf.classes_)
They are identical. But when I run:
print(model.decision_function(X_test))
print(clf.decision_function(X_test))
The result is:
array([-0.17743171, -0.10924828, -1.00177102, -0.88501918, -0.93835179])
array([0.68707567, 0.28820641, 0.00613415, 0.01067078, 0.00791299])
That is - "model" get highest "weight" for class number 2 while "clf" get highest probability for class number 1. These two models thus give different predictions. My interpretation of Stackoverflow is that "clf" should just be the sigmoid of the decision_function-values from "model", and thus the class with the highest weight (from model) should also have the highest probability (from clf). This seem to not be the case, so if someone know why - please explain to me!
EDIT: Here are also the .get_params() output from model and clf:
print(model.get_params())
print(clf.get_params())
{'C': 1.0,
'class_weight': None,
'dual': True,
'fit_intercept': True,
'intercept_scaling': 1,
'loss': 'squared_hinge',
'max_iter': 1000,
'multi_class': 'ovr',
'penalty': 'l2',
'random_state': None,
'tol': 0.0001,
'verbose': 0}
{'base_estimator__C': 1.0,
'base_estimator__class_weight': None,
'base_estimator__dual': True,
'base_estimator__fit_intercept': True,
'base_estimator__intercept_scaling': 1,
'base_estimator__loss': 'squared_hinge',
'base_estimator__max_iter': 1000,
'base_estimator__multi_class': 'ovr',
'base_estimator__penalty': 'l2',
'base_estimator__random_state': None,
'base_estimator__tol': 0.0001,
'base_estimator__verbose': 0,
'base_estimator': LinearSVC(C=1.0, class_weight=None, dual=True, fit_intercept=True,
intercept_scaling=1, loss='squared_hinge', max_iter=1000,
multi_class='ovr', penalty='l2', random_state=None, tol=0.0001,
verbose=0),
'cv': 'prefit',
'method': 'sigmoid'}
