2

While running sagemaker in local mode. I am experimenting with an inference endpoint in local mode using docker container. But as soon as my model.tar.gz file exceeds a certain size i.e. around 200 mb, the deployment fails and returns the error: RuntimeError: Giving up, endpoint: didn't launch correctly

When I deploy it on a sagemaker instance, it works fine. Do you know if there is something I could do, perhaps some docker setting I could change to make sure that the local deployment also works with the larger model.tar.gz?

andyk
  • 25
  • 2

1 Answers1

2

Instead of using SageMaker SDK Local Mode, you can also run vanilla docker commands yourself to imitate the hosted environment:

Such as:

  1. Start "local endpoint"
image=$1

docker run -v $(pwd)/test_dir:/opt/ml -p 8080:8080 --rm ${image} serve
  1. Invoke "local endpoint"
payload=$1
content=${2:-text/csv}

curl --data-binary @${payload} -H "Content-Type: ${content}" -v http://localhost:8080/invocations

https://github.com/aws/amazon-sagemaker-examples/tree/main/advanced_functionality/scikit_bring_your_own/container/local_test

Marc Karp
  • 949
  • 4
  • 6