Trained and loaded a sklearn model but cannot access the model to perform prediction

Question

I've trained a model and saved it using a class method. When doing evaluation, I wanted to load the trained model and predict on my test file; but I'm not able to even use my loaded model. I checked the type of the loaded model and it's a method but I'm not sure if it's the correct type to use.

Also, I'm not sure if my save_model method has saved the fit model with the trained model and I was thinking maybe that is the reason I cannot access my loaded model. Sorry if I'm not very clear, please refer to my code below -- it'd be much clearer as to what I mean.

I've searched all over the internet and all of the solutions don't seem to help.. So any help will be very much appreciated!!!!

The save model method

# POS class
def __init__(self):
    self.vec = DictVectorizer()
    self.model = LinearSVC()

def fit_and_report(self, X, Y):
    X = self.vec.fit_transform(X)
    self.model.fit(X, Y)

def save_model(self, output_file):
    with open(output_file, "wb") as outfile:
        pickle.dump(self, outfile)

When first training the model, I've done the following.

X, Y = pre_process.load_dataset('my_train_file')
X, Y = pre_process.prepare_data_for_training(X, Y)
my_model = function_in_tagger.POS() #call the POS class from a separate script
my_model.fit_and_report(X, Y) # I assumed the self.model.fit(X, Y) in the method is also saved to the model???
my_model.save_model('my_model_file')

Load and perform prediction

X_test, Y_test = pre_process.load_dataset('my_test_file') # it returns X and Y (both are list)
loaded_model = pickle.load(open('my_model_file', 'rb'))
y_predict = loaded_model.predict(X_test)

The error I got is

y_predict = loaded_model.predict(Y_test)

AttributeError: 'POS' object has no attribute 'predict'

I also tried getting the score of the trained model, but got the same error

score = loaded_model.score(X_test, Y_test)

AttributeError: 'POS' object has no attribute 'score'

---

New edit

I've tried to transform my test data using DictVectorizer, but it gives me the an attribute error ('DictVectorizer' object has no attribute 'feature_names_')

vec = DictVectorizer()
vec.transform(X_test)

Then I tried to fit the test data before transforming, it then gave me no error message. But from what I understand, I should not be fitting my test data, only transform.

When using both the fit and transform method, it still gives me the following error though

y_predict = loaded_model.predict(Y_test)

AttributeError: 'POS' object has no attribute 'predict'

Answer 1

Your error is originating from your test data. You are suppose to transform the test data before subjecting to your trained model. Your model was trained on transformed data not list See below example based on text classifcation;

#Loading the trained model

with open('Rf Classifier', 'rb') as training_model: Rf_model = pickle.load(training_model))

After Loading the model, transform the test data in my case it is in form of a data frame. Here in since I was using more than one variable I am using DataframeMapper library to transform the test data

test_data=mapper.transform(df_test) #df_test refers to the test data which is a dataframe

There after, you predict your transformed/vectorized data using the trained model as shown below;

df_test["class"] = Rf_model.predict(test_data) # this section creates a new column for predicted class