SageMaker Experiments: UnpicklingError: invalid load key, '\x1f'

Question

I am new to Data Science and have been trying to use SageMaker Experiments to create an extremely simple model. Using SageMaker Experiments, I trained a model using a CSV dataset. The model was output to S3, and I am now trying to load that model into a Jupyter Notebook and run a batch transform test on it.

However, I am getting an error when trying to use the following block of code provided in the SageMaker docs that is supposed to help me import the model. source

import pickle as pkl
import tarfile
import xgboost

t = tarfile.open('model.tar.gz', 'r:gz')
t.extractall()

model = pkl.load(open(model_file_path, 'rb'))

The error is occurring at pkl.load:

 ---------------------------------------------------------------------------
UnpicklingError                           Traceback (most recent call last)
<ipython-input-39-81639df86023> in <module>
      6 t.extractall()
      7 
----> 8 model = pkl.load(open(model_file_path, 'rb'))
      9 
     10 

UnpicklingError: invalid load key, '\x1f'.

Note: model_file_path is 'model.tar.gz'

Because I am so new at this, I do not know if it is an error with what I have done, or if there is something about how SageMaker Experiments does something that I am missing. I have tried referencing other stackoverflow posts that contain this error, but they don't seem to directly apply (to my understanding) to my error. (this and this)

Any expert advice on SageMaker Experiments or the SageMaker v2 SDK would be extremely helpful to this newbie trying to break into the space. Thank you!

score 0 · Answer 1 · answered Jul 22 '21 at 18:31

It does not look like your issue is being caused by SageMaker Experiments. This is either an issue with your model.tar.gz file itself or the way you are loading it using pkl.

model.tar.gz should contain the pickle file for the model itself. You can then extract the pickle file from model.tar.gz and load that into your model object.

Alternatively, you could use the XGBoostModel class to create a model from the model.tar.gz file itself. Refer to the following docs for more details on loading a pre-trained XGBoost Model.

I work for AWS & my opinions are my own

SageMaker Experiments: UnpicklingError: invalid load key, '\x1f'

1 Answers1