How to revise this code for implementing Smote Oversampling and Cross Validation pipeline to multiclass classification problem?

Question

imba_pipeline = make_pipeline(SMOTE(random_state=42), 
                              DecisionTreeClassifier())
cross_val_score(imba_pipeline, X_train1, y_train1, scoring='recall', cv=kf)

When I run this, it gives below error:

ValueError: Target is multiclass but average='binary'. Please choose another average setting, one of [None, 'micro', 'macro', 'weighted'].

    Out[102]:
    array([nan, nan, nan, nan, nan])

Does this answer your question? [Facing ValueError: Target is multiclass but average='binary'](https://stackoverflow.com/questions/52269187/facing-valueerror-target-is-multiclass-but-average-binary) — G. Anderson, Jan 03 '22 at 17:03

score 0 · Answer 1 · answered Jan 03 '22 at 17:15

If you use scoring='recall' , it calls sklearn.metrics.recall_score with the default, average = "binary" . One way is to create a custom scoring:

from sklearn.datasets import load_iris
from sklearn.tree import DecisionTreeClassifier
from imblearn.over_sampling import SMOTE
from imblearn.pipeline import make_pipeline
from sklearn.model_selection import cross_val_score
from sklearn.metrics import recall_score
from sklearn.metrics import make_scorer

X,y = load_iris(return_X_y=True)

imba_pipeline = make_pipeline(SMOTE(random_state=42), 
                              DecisionTreeClassifier())

multi_recall = make_scorer(recall_score, average="micro")

cross_val_score(imba_pipeline, X, y, scoring=multi_recall, cv=5)

array([0.96666667, 0.96666667, 0.9       , 0.93333333, 1.        ])

How to revise this code for implementing Smote Oversampling and Cross Validation pipeline to multiclass classification problem?

1 Answers1