13

I have a sagemaker instance up and running and I have a few libraries that I frequently use with it but each time I restart the instance they get wiped and I have to reinstall them. Is it possible to install my libraries to one of the anaconda environments and have the change remain?

Adrix
  • 401
  • 1
  • 3
  • 11

2 Answers2

12

The supported way to do this for Sagemaker notebook instances is with Lifecycle Configurations.

You can create an onStart lifecycle hook that can install the required packages into the respective Conda environments each time your notebook instance starts.

Please see the following blog post for more details

https://aws.amazon.com/blogs/machine-learning/customize-your-amazon-sagemaker-notebook-instances-with-lifecycle-configurations-and-the-option-to-disable-internet-access/

Jaipreet
  • 302
  • 1
  • 6
2

When creating you model, you can specify the requirements.txt as an environment variable.

For Eg.

env = {
    'SAGEMAKER_REQUIREMENTS': 'requirements.txt', # path relative to `source_dir` below.
}
sagemaker_model = TensorFlowModel(model_data = 's3://mybucket/modelTarFile,
                                  role = role,
                                  entry_point = 'entry.py',
                                  code_location = 's3://mybucket/runtime-code/',
                                  source_dir = 'src',
                                  env = env,
                                  name = 'model_name',
                                  sagemaker_session = sagemaker_session,
                                 )

This would ensure that the requirements file is run after the docker container is created, before running any code on it.

Raman
  • 643
  • 5
  • 6
  • If I want to have a library persist in a jupyter notebook instance is that possible with a similar method? – Adrix Jul 01 '18 at 15:20
  • Yes you can. Make sure you put the 'requirements.txt' in a sub directory of the Jupyter notebook and mention that directory in the source_directory argument as mention in the above answer. – Raman Jul 02 '18 at 17:59