2

I trained some data science models with scikit learn from v0.19.1. The models are stored in a pickle file. After upgrading to latest version (v0.23.1), I get the following error when I try to load them:

 File "../../Utils/WebsiteContentSelector.py", line 100, in build_page_selector
   page_selector = pickle.load(pkl_file)
 AttributeError: Can't get attribute 'DeprecationDict' on <module 'sklearn.utils.deprecation' from '/usr/local/lib/python3.6/dist-packages/sklearn/utils/deprecation.py'>

Is there a way to upgrade without retraining all my models (which is very expensive)?

Julia Meshcheryakova
  • 3,162
  • 3
  • 22
  • 42
Robycool
  • 1,104
  • 2
  • 14
  • 26

3 Answers3

0

You used a new version of sklearn to load a model which was trained by an old version of sklearn.

So, the options are:

  • Retrain the model with current version of sklearn if you have the training script and data
  • Or fall back to the lower sklearn version reported in the warning message
Xuan Zhang
  • 46
  • 2
  • 2
    Your answer to "How can I do X without doing Y" is "By doing Y?" – Pranav Hosangadi Aug 04 '20 at 22:10
  • I'm suggesting solutions to your general question "how can I do X". You may try downgrade your running sklearn version, if it won't affect other part of your system. The error message clearly indicated the data structure gap between 2 sklearn versions, which is hard to fill without retraining. Good luck. – Xuan Zhang Aug 07 '20 at 16:47
0

Depending on the kind of sklearn model used, if the model is simple regression model, what is probably needed is to get the actual weights and bias (or intercept) values.

You can check these values in your model:

model.classes_
model.coef_
model.intercept_

they are of numpy type and can be pickled easily. Also, you need to get the same parameters passed to the model construction. For example:

tol
max_iter

and so on. With this, in the upgraded version, the same model created with the same parameters can read the weights and intercept.

In this way, no re-training is needed and you can use the upgrade sklearn.

jefka19
  • 1
0

When lib versions are not backward compatible you can do the following:

  1. Downgrade sklearn back to the original version
  2. Load each model, extract and store its coefficients (which are model-specific - check documentation)
  3. Upgrade sklearn, load coefficients and init models with them, save models

Related question.

Julia Meshcheryakova
  • 3,162
  • 3
  • 22
  • 42