When Amazon SageMaker Neo is supported for your ML framework and the EC2 instance you want to use (see this page), should you always compile your model as a best practice to get a better throughput and latency?
Or are there cases when compiled model is not a good idea? I'm asking the question because I see a lot of notebook examples where the trained model is deployed as uncompiled and I'm wondering if it is just because of the simplicity or if there is another reason?
Thank you in advance,
