Precision, F-score and Recall High on first prediction

Question

I have printed the classification report of my SVM model predicting on binary classes, but it scored high (over 95%) on the first prediction, I know that it is good when it printed high values, but I need to perform feature selection after this, do you think this is normal? And what kind of feature selection does suitable for binary classification?

Answer 1

do you think this is normal?

It depends on your data. If your data are completely linearly separable, then almost all linear classifiers will perform well.

If all precision, recall and F score are high, and they are high enough for you, then maybe you don't have to go further.

You can see the feature importance, then you can read this article: Determining the most contributing features for SVM classifier in sklearn

And if you use another algorithm, the feature importance maybe a little bit different.