3

I use compressed pickle to save the results from sklearn gridsearch using the following code.

import pickle
import bz2
from sklearn.model_selection import RandomizedSearchCV

search = RandomizedSearchCV(estimator, param_distributions=param_dist,
                                   cv = cv, 
                                   n_jobs = 6
                                   )
with bz2.open('gridsearch.bz2', 'wb') as f:
        pickle.dump(search, f)

Then I use the following code to get the search object.

with bz2.open('gridsearch.bz2', 'rb') as f:
    search = pickle.load(f)

The pickle load works when I run in virtual environment in terminal but it report the following error if I run the code in jupyter lab. I checked the version of pickle. Both the virtual environment and jupyter lab use pickle 4.0.

---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-11-06b38588a6ad> in <module>
---> 23 with bz2.open('gridsearch.bz2', 'rb') as f:
     24    search = pickle.load(f)

~/.pyenv/versions/3.7.4/lib/python3.7/site-packages/scipy/stats/_distn_infrastructure.py in __setstate__(self, state)
    623 
    624     def __setstate__(self, state):
--> 625         ctor_param, r = state
    626         self.__init__(**ctor_param)
    627         self._random_state = r

ValueError: too many values to unpack (expected 2)

BTW, I have also tried to use .pickle file instead of bz2. This problem still occurs. Any comment and suggestions would be appreciated.

kaidi
  • 31
  • 2

0 Answers0