5

I have a model.pkl file which is pre-trained and all other files related to the ml model. I want it to deploy it on the aws sagemaker. But without training, how to deploy it to the aws sagmekaer, as fit() method in aws sagemaker run the train command and push the model.tar.gz to the s3 location and when deploy method is used it uses the same s3 location to deploy the model, we don't manual create the same location in s3 as it is created by the aws model and name it given by using some timestamp. How to put out our own personalized model.tar.gz file in the s3 location and call the deploy() function by using the same s3 location.

tarun mittal
  • 331
  • 4
  • 13
  • Does this answer your question? [How to run a python file inside a aws sagemaker using dockerfile](https://stackoverflow.com/questions/58300841/how-to-run-a-python-file-inside-a-aws-sagemaker-using-dockerfile) – Pramesh Bajracharya Nov 15 '19 at 10:09

1 Answers1

7

All you need is:

  1. to have your model in an arbitrary S3 location in a model.tar.gz archive
  2. to have an inference script in a SageMaker-compatible docker image that is able to read your model.pkl, serve it and handle inferences.
  3. to create an endpoint associating your artifact to your inference code

When you ask for an endpoint deployment, SageMaker will take care of downloading your model.tar.gz and uncompressing to the appropriate location in the docker image of the server, which is /opt/ml/model

Depending on the framework you use, you may use either a pre-existing docker image (available for Scikit-learn, TensorFlow, PyTorch, MXNet) or you may need to create your own.

  • Regarding custom image creation, see here the specification and here two examples of custom containers for R and sklearn (the sklearn one is less relevant now that there is a pre-built docker image along with a sagemaker sklearn SDK)
  • Regarding leveraging existing containers for Sklearn, PyTorch, MXNet, TF, check this example: Random Forest in SageMaker Sklearn container. In this example, nothing prevents you from deploying a model that was trained elsewhere. Note that with a train/deploy environment mismatch you may run in errors due to some software version difference though.

Regarding your following experience:

when deploy method is used it uses the same s3 location to deploy the model, we don't manual create the same location in s3 as it is created by the aws model and name it given by using some timestamp

I agree that sometimes the demos that use the SageMaker Python SDK (one of the many available SDKs for SageMaker) may be misleading, in the sense that they often leverage the fact that an Estimator that has just been trained can be deployed (Estimator.deploy(..)) in the same session, without having to instantiate the intermediary model concept that maps inference code to model artifact. This design is presumably done on behalf of code compacity, but in real life, training and deployment of a given model may well be done from different scripts running in different systems. It's perfectly possible to deploy a model with training it previously in the same session, you need to instantiate a sagemaker.model.Model object and then deploy it.

Olivier Cruchant
  • 3,747
  • 15
  • 18
  • is there a step-by-step detailed instruction of deploying a pre-trained ensorflow tmodel to SageMaker? – wawawa Nov 22 '20 at 15:49
  • https://aws.amazon.com/blogs/machine-learning/deploy-trained-keras-or-tensorflow-models-using-amazon-sagemaker/ – Olivier Cruchant Nov 22 '20 at 16:44
  • Hi Olivier thanks for sharing, in this post it's using a .h5 formatted file and using a notebook instance to deploy, I converted my model from .h5 to a .tar.gz file, and I have a python script to create the model, just wondering if there's a way to use them to deploy the model to SageMaker? – wawawa Nov 22 '20 at 16:56
  • you have 2 options: (A) use SageMaker TFServing (hosting) integration. However this doesn't work with .h5, only .pb so you'll need to convert from h5 to pb. (B) use the SageMaker Scikit container. Create a model_fn than can read your h5 and return a model in memory, then input_fn, predict_fn and output_fn that can manage your payload and prediction. And deploy with a requirements.txt so that you can install TensorFlow https://sagemaker.readthedocs.io/en/stable/frameworks/sklearn/using_sklearn.html#deploy-a-scikit-learn-model. If you want to deploy to GPU (A) is the best option. – Olivier Cruchant Nov 22 '20 at 18:37
  • Hi as far as I know SageMaker can only understand .tar.gz format, so can I create an endpoint by using model.tar.gz file? – wawawa Nov 23 '20 at 11:26
  • SageMaker needs model.tar.gz, but what you put inside the tar.gz is your choice and can be anything. For SageMaker TFServing integration, it must be a TensorFlow SavedModel. – Olivier Cruchant Nov 23 '20 at 11:34
  • @OlivierCruchant Can you please take a look at this? https://stackoverflow.com/questions/65434323/how-to-deploy-a-pre-trained-model-using-aws-sagemaker-notebook-instance – Muhammad Arsalan Hassan Dec 24 '20 at 05:29