Stack Overflow
Stack Overflow

Questions tagged [amazon-sagemaker-compilers]

10 questions
9
votes
4 answers

get_execution_role() sagemaker: UnknownServiceError: Unknown service: 'sagemaker-metrics'. Valid service names are: accessanalyzer

I am trying this code to get iam role programmatically. from sagemaker import get_execution_role get_execution_role() It's giving the following error. UnknownServiceError Traceback (most recent call…
iamabhaykmr
  • 1,803
  • 3
  • 24
  • 49
2
votes
1 answer

AWS SageMaker Neo v.s. Native Optimized Runtimes

What is the advantage of SageMaker Neo vs the specialized native runtimes that every ML accelerator provides, such as NVIDIA TensorRT, Intel OpenVINO, DeepView RT, CoreML, ArmNN, etc.? I understand that Neo uses some of these frameworks, like…
juvchan
  • 6,113
  • 2
  • 22
  • 35
2
votes
0 answers

SageMaker Neo compilation for PyTorch BERT

I'm trying to optimize a BERT model with SageMaker Neo that is pre-trained using PyTorch, but I'm getting the following error: ClientError: InputConfiguration: Exactly one .pt or .pth file is allowed for PyTorch models. My model.tar.gz is only…
juvchan
  • 6,113
  • 2
  • 22
  • 35
2
votes
1 answer

Which techniques are used by SageMaker Neo for model optimizations

Does SageMaker Neo (SageMaker compilation job) use any techniques for model optimization? Are there any compression techniques used (distillation, quantization etc) to reduce the model size? I found some description here…
ryfeus
  • 323
  • 2
  • 6
1
vote
1 answer

SageMaker Neo Compilation - Unable to Neo Compile for FP16 and INT8 precision

I'm trying to Neo compile a Pytorch YoloV5 Large model for edge deployment on an Nvidia Jetson Xavier NX device. I'm able to do it using the default settings for FP32 precision but I'm unable to do it for FP16 or INT8 precision. I have tried passing…
Shitij Mathur
  • 385
  • 2
  • 10
0
votes
1 answer

What if i say ” sagemaker async is nothing but a task queue which use ML type instances.”

If we run All the inference code within the celery task or another task queue like rabbit MQ, we will get the same performance ( if we ignore the impact of ML type instance ). What are the major benefits we will get if we are using sagemaker async…
Bajaj_Dk
  • 65
  • 6
0
votes
0 answers

Amazon SageMaker Neo for EC2 instances

When Amazon SageMaker Neo is supported for your ML framework and the EC2 instance you want to use (see this page), should you always compile your model as a best practice to get a better throughput and latency? Or are there cases when compiled model…
Cyril Bandolo
  • 16
  • 1
0
votes
0 answers

Can I deployed a SageMaker Neo compile XGBoost model on-prem?

My team would like to train and compile our model in the cloud, then deploy it on-perm.I know that Neo provides an optimized runtime. Only problem is we have our own rutime. Does Neo compile the decision tree itself? Also, we aim to work with intel…
Cyril Bandolo
  • 16
  • 1
0
votes
0 answers

Can SageMaker Neo compile a XGBoost model targeted Windows platform

We are building a prototype for a customer who would like to run offline inference using XGBoost on Windows Server runtime. We are currently exploring the option of compiling the model using SageMaker Neo. From the Neo page it says it supports…
Cyril Bandolo
  • 16
  • 1
0
votes
0 answers

Sagemaker Neo vs ONNX and Torchscript

I have read about Sagemaker Neo, ONNX, and Torchscript. I would like to know which one of these is better to use compared to the others.
Cyril Bandolo
  • 16
  • 1