Problem loading ML model saved using joblib/pickle

Question

I saved a jupyter notebook .pynb file to .pickle format using joblib.

My ML model is built using pandas, numpy and the statsmodels python library.

I saved the fitted model to a variable called fitted_model and here is how I used joblib:

from sklearn.externals import joblib
# Save RL_Model to file in the current working directory

joblib_file = "joblib_RL_Model.pkl"  
joblib.dump(fitted_model, joblib_file)

I get this as output:

['joblib_RL_Model.pkl']

But when I try to load from file, in a new jupyter notebook, using:

# Load from file
joblib_file = "joblib_RL_Model.pkl"  
joblib_LR_model = joblib.load(joblib_file)
joblib_LR_model

I only get this back: <statsmodels.tsa.holtwinters.HoltWintersResultsWrapper at 0xa1a8a0ba8>

and no model. I was expecting to see the model load there and see the graph outputs as per original notebook.

ashraful16 · Accepted Answer · 2020-08-29T18:48:37.913

3

Use with open, it is better because, it automatically open and close file. Also with proper mode.

with open('joblib_RL_Model.pkl', 'wb') as f:
    pickle.dump(fitted_model, f)

                                    
with open('joblib_RL_Model.pkl', 'rb') as f:
    joblib_LR_model = pickle.load(f)

And my implementation in Colab is here. Check it.

edited Aug 29 '20 at 18:48

answered Aug 29 '20 at 17:52

ashraful16

2,742
3
11
32

import library using import pickle – ashraful16 Aug 29 '20 at 18:05
imported it, the error is now: EOFError: Ran out of input – cheznead Aug 29 '20 at 18:07
1

Check your model if it is blank. For more, https://stackoverflow.com/questions/24791987/why-do-i-get-pickle-eoferror-ran-out-of-input-reading-an-empty-file – ashraful16 Aug 29 '20 at 18:10
ok, now error is: name 'fitted_model' is not defined – cheznead Aug 29 '20 at 18:12
fitted_model is the desired model that you want to save. – ashraful16 Aug 29 '20 at 18:14
yes, I made a mistake and had both your commands in the same new file. I now have dump method in original and load in new file. But when I run new file, nothing happens. – cheznead Aug 29 '20 at 18:16
You can share a public Colab file on your work. It will better. – ashraful16 Aug 29 '20 at 18:20
Let us [continue this discussion in chat](https://chat.stackoverflow.com/rooms/220674/discussion-between-cheznead-and-ashraful). – cheznead Aug 29 '20 at 18:27

siva · Answer 2 · 2020-08-29T18:24:22.990

1

you can use more quantifiable package which is pickle default package for python to save models

you can use the the following function for the saving of ML Models

import pickle

def save_model(model):
    pickle.dump(model, open("model.pkl", "wb"))

template for function would be

import pickle

def save_model(model):
    pickle.dump(model, open(PATH_AND_FILE_NAME_TO_BE_SAVED, "wb"))

to load the model when saved it from pickle library you can follow the following function

def load_model(path):
    return pickle.load(open(path, 'rb'))

Where path is the path and name to file where model is saved to.

Note:

This would only work for basic ML Models and PyTorch Models, it would not work for Tensorflow based models where you need to use

model.save(PATH_TO_MODEL_AND_NAME)

where model is of type tensorflow.keras.models

edited Aug 29 '20 at 18:24

answered Aug 29 '20 at 18:00

siva

21
4

what would you use then to reload it? – cheznead Aug 29 '20 at 18:06
damn, same error as with the other method. 'ran out of input'. I did this: def load_model(): pickle.load(open("model.pkl", 'rb')) in new file – cheznead Aug 29 '20 at 18:20
1

can you please check with the edited answer as i have forgot return in the function, and please do do it like pickle.load(open("./model.pkl", "rb")) – siva Aug 29 '20 at 18:25

Problem loading ML model saved using joblib/pickle

2 Answers2