AWS - ECS load S3 files in entrypoint script

Question

Hi all!

Code: (entrypoint.sh)

printenv
CREDENTIALS=$(curl -s "http://169.254.170.2$AWS_CONTAINER_CREDENTIALS_RELATIVE_URI")

ACCESS_KEY_ID=$(echo "$CREDENTIALS" | jq .AccessKeyId)
SECRET_ACCESS_KEY=$(echo "$CREDENTIALS" | jq .SecretAccessKey)
TOKEN=$(echo "$CREDENTIALS" | jq .Token)

export AWS_ACCESS_KEY_ID=$ACCESS_KEY_ID
export AWS_SECRET_ACCESS_KEY=$SECRET_ACCESS_KEY
export AWS_SESSION_TOKEN=$TOKEN

aws s3 cp s3://BUCKET/file.txt /PATH/file.txt

Problem:

I'm trying to fetch AWS S3 files to ECS inspired by: AWS Documentation (But I'm fetching from S3 directly, not throught VPC endpoint)
I have configured bucket policy & role policy (that is passed in taskDefinition as taskRoleArn & executionRoleArn)
Locally when I'm fetching with aws cli and passing temporary credentials (that I logged in ECS with printenv command in entrypoint script) everything works fine. I can save files on my pc.
On ECS I have error:

fatal error: An error occurred (403) when calling the HeadObject operation: Forbidden

Where can I find solution? Someone had similar problem?

Answer 1

Frist thing, If you are working inside AWS, It strongly recommended to use AWS ECS service role or ECS task role or EC2 role. you do need to fetch credentials from metadata.

But seems like the current role does have permission to s3 or the entrypoint not exporting properly the Environment variable.

If your container instance has already assing role then do not need to export Accesskey just call the aws s3 cp s3://BUCKET/file.txt /PATH/file.txt and it should work.

IAM Roles for Tasks

With IAM roles for Amazon ECS tasks, you can specify an IAM role that can be used by the containers in a task. Applications must sign their AWS API requests with AWS credentials, and this feature provides a strategy for managing credentials for your applications to use, similar to the way that Amazon EC2 instance profiles provide credentials to EC2 instances. Instead of creating and distributing your AWS credentials to the containers or using the EC2 instance’s role, you can associate an IAM role with an ECS task definition or RunTask API operation.

So the when you assign role to ECS task or ECS service your entrypoint will be that simple.

printenv
aws s3 cp s3://BUCKET/file.txt /PATH/file.txt

Also, your export will not work as you are expecting, the best way to pass ENV to container form task definition, export will not in this case.

I will suggest assigning role to ECS task and it should work as you are expecting.