How to get the support vectors when using LinearSVC?

Question

I am using GridSearchCV and would like to save the support vectors as follows:

np.save("support_vectors.npy", gs_cv.best_estimator_.named_steps['svm'].support_vectors_)

But I get this error:

  File "./improved_grid_search.py", line 500, in <module>
    np.save("support_vectors.npy", gs_cv.best_estimator_.named_steps['svm'].support_vectors_)
AttributeError: 'LinearSVC' object has no attribute 'support_vectors_'

I checked and indeed LinearSVC doesn't seem to have this attribute. Where then are they saved?

I am using the following Pipe:

search_spaces = []
if ovr:
        if 'linear' in kernel_lst:
            search_spaces.append({'svm': [OneVsRestClassifier(LinearSVC())], # optmized version of linear
                                  'svm__estimator__C': c})
        if 'poly' in kernel_lst:
            search_spaces.append({'svm': [OneVsRestClassifier(SVC(kernel='poly', cache_size=cache_size))],
                                  'svm__estimator__degree': degree,
                                  'svm__estimator__C': c})
        if 'rbf' in kernel_lst:
            search_spaces.append({'svm': [OneVsRestClassifier(SVC(kernel='rbf', cache_size=cache_size))],
                                  'svm__estimator__gamma': gamma,
                                  'svm__estimator__C': c})
else:
        if 'linear' in kernel_lst:
            search_spaces.append({'svm': [LinearSVC()], # optmized version of linear
                                  'svm__C': c})
        if 'poly' in kernel_lst:
            search_spaces.append({'svm': [SVC(kernel='poly', cache_size=cache_size)],
                                  'svm__degree': degree,
                                  'svm__C': c})
        if 'rbf' in kernel_lst:
            search_spaces.append({'svm': [SVC(kernel='rbf', cache_size=cache_size)],
                                  'svm__gamma': gamma,
                                  'svm__C': c})

svm_pipe = Pipeline([('svm', DummyClassifier())])

And grid search is done this way:

grid_cv_object = GridSearchCV(
        estimator = svm_pipe,
        param_grid = search_spaces,
        cv = cv_splits,
        scoring = make_scorer(matthews_corrcoef), # a callable returning single value, binary and multiclass labels are supported
        n_jobs = -1, # use all processors
        verbose = 10,
        refit = refit,
        return_train_score = True
    )

score 0 · Answer 1 · answered Apr 23 '22 at 10:11

0

LinearSVC optimizes a linear model (like LogisticRegression or Lasso) with a complexity penalty via some gradient-based method, without regard for support vectors. I'm afraid only SVC and NuSVC provide support_vectors_.

answered Apr 23 '22 at 10:11

Sanjar Adilov

1,039
7
15

desertnaut · Answer 2 · 2022-04-27T15:29:56.973

As you have already discovered yourself, LinearSVC does not have a support_vectors_ attribute, only coef_ and intercept_ ones.

However, according to the documentation, LinearSVC is:

Similar to SVC with parameter kernel=’linear’, but implemented in terms of liblinear rather than libsvm, so it has more flexibility in the choice of penalties and loss functions and should scale better to large numbers of samples.

So, if you don't have performance issues, you can end up with your desired support_vectors_ simply by replacing LinearSVC() with the largely equivalent SVC(kernel='linear').

For more details, see Under what parameters are SVC and LinearSVC in scikit-learn equivalent? (tl;dr: LinearSVC is actually not a linear SVM).

How to get the support vectors when using LinearSVC?

2 Answers2