The most import features for a SVM text classification with Python and scikit-learn

Question

I'm new to artificial intelligence. I'm working with the SVM algorithm and ran this Python script to train/predict if an email is spam or not. The script works:

import pandas as pd
from sklearn.model_selection import train_test_split
from sklearn.feature_extraction.text import CountVectorizer
from sklearn import svm

# dependencies
# pip install pandas
# pip install -U scikit-learn

spam = pd.read_csv('Cartel1.csv')
z = spam['v2']
y = spam["v1"]

#Splitting our data into training and testing.
z_train, z_test,y_train, y_test = train_test_split(z,y,test_size = 0.2)
#Converting text into integer using CountVectorizer
cv = CountVectorizer()
features = cv.fit_transform(z_train)
svm = svm.SVC()
svm.fit(features,y_train)
features_test = cv.transform(z_test)

comment =["Sexy free Call and text messages on 08002986030"]
vect= cv.transform(comment) 
print("This comment: ", comment, " is: ", svm.predict(vect))#spam

comment2 =["Hi there, I am emailing you today to let you know we have created a new task for you."]
vect2= cv.transform(comment2) 
print("This comment: ", comment2, " is: ", svm.predict(vect2))#ham --no spam
#print(model.score(features_test,y_test))

But I was hoping that I could inspect the model to get the most frequent words classified as "spam" and "ham." I would like to get a result similar to this: Determining the most contributing features for SVM classifier in sklearn

I would like to get the most frequent words classified as spam or ham.

Answer 1

The linked question + the example here gets pretty close.

Let's rewrite this into a minimal reproducible example though. SVM assigns importance to features based on how well they discriminate between a positive and negative feature set, so it will be more helpful if we can show some features that are ambiguous whether they're spam/ham:

from sklearn.feature_extraction.text import CountVectorizer
from sklearn.svm import SVC
import numpy as np

raw_data = [
    "hello sexy free call and text messages on 08002986030",
    "hello see my text with a new task",
    "this is your boss please text me",
]
y = np.array([1.0, 0.0, 0.0])

vectorizer = CountVectorizer()
X = vectorizer.fit_transform(raw_data)

Notice that the word "text" appears in all three examples.

We can fit the SVM on the vectorized X and y using the linear kernel suggested in the other question:

svm = SVC(kernel="linear")
svm.fit(X, y)

Visualizing involves (1) extracting SVM coefficients, (2) getting the words, (3) sorting and plotting:

coefs = svm.coef_.toarray().flatten()
words = vectorizer.get_feature_names_out()

coefs, words = zip(*sorted(zip(coefs, words), key=lambda x: x[0], reverse=True))
plt.barh(words, coefs)
plt.show()

Notice that the word "text" has a coefficient of 0.0, implying that it is not helpful in discriminating between spam/ham:

For larger datasets the set of coefficients and words may stretch into the thousands or tens-of-thousands. In that case, you might apply the sorting method but then sample the top-10 and bottom-10 cases.