How do you specify AWS credentials when running AWS CLI from a Dockerfile in an AWS SAM pipeline?

Question

I have an app using:

SAM
AWS S3
AWS Lambda based on Docker
AWS SAM pipeline
Github function

In the Dockerfile I have:

RUN aws s3 cp s3://mylambda/distilBERT distilBERT.tar.gz

Resulting in the error message:

Step 6/8 : RUN aws s3 cp s3://mylambda/distilBERT distilBERT.tar.gz
 ---> Running in 786873b916db
fatal error: Unable to locate credentials
Error: InferenceFunction failed to build: The command '/bin/sh -c aws s3 cp s3://mylambda/distilBERT distilBERT.tar.gz' returned a non-zero code: 1

I need to find a way to store the credential in a secured manner. Is it possible with GitHub secrets or something?

Thanks

Ervin Szilagyi · Answer 1 · 2021-12-10T22:08:53.100

2

Docker by default does not have access to the .aws folder running on the host machine. You could either pass the AWS credentials as environment variables to the Docker image:

ENV AWS_ACCESS_KEY_ID=AKIAIOSFODNN7EXAMPLE
ENV AWS_SECRET_ACCESS_KEY=...

Keep in mind, hardcoding AWS credentials in a Dockerfile is a bad practice. In order to avoid this, you can pass the environment variables at runtime with using docker run -e MYVAR1 or docker run --env MYVAR2=foo arguments. Other solution would be to use an .env file for the environment variables.

A more involved solution would be to map a volume for the ~/.aws folder from the host machine in the Docker image.

edited Dec 10 '21 at 22:08

answered Dec 10 '21 at 21:36

Ervin Szilagyi

14,274
2
25
40

Wouldn't that be a bad practice since the AWS credentials are now exposed in the dockerfile? And also it would be a pain to change them later(if required)? – Sanil Khurana Dec 10 '21 at 21:53
1

Hardcoding them in Dockerfile it would be a bad practice. You can pass environment variable at runtime `docker run -e MYVAR1 --env MYVAR2=foo`. Or you can use an `env-file`. Moreover, you can assume a role and pass those secrets. – Ervin Szilagyi Dec 10 '21 at 22:00
1

In fact, I think is important to update my answer with these, since they are important from security perspective. Thanks @SanilKhurana – Ervin Szilagyi Dec 10 '21 at 22:10

score 2 · Answer 2 · answered Dec 10 '21 at 21:59

My solution may be a bit longer but I feel it solves your problem, and

It does not expose any secrets
It does not require any manual work
It is easy to change your AWS keys later if required.

Steps:

You can add the environment variables in Github actions(since you already mentioned Github actions) as secrets.
In your Github CI/CD flow, when you build the Dockerfile, you can create a aws credentials file.

      - name: Configure AWS credentials
        echo "
        [default]
        aws_access_key_id = $ACCESS_KEY
        aws_secret_access_key = $SECRET_ACCESS_KEY
        " > credentials
        with:
          ACCESS_KEY: ${{ secrets.AWS_ACCESS_KEY_ID }}
          SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }}

In your Dockerfile, you can add instructions to COPY this credentials file and store it

COPY credentials credentials
RUN mkdir ~/.aws
RUN mv credentials ~/.aws/credentials

Changing your credentials requires just changing your github actions.

How do you specify AWS credentials when running AWS CLI from a Dockerfile in an AWS SAM pipeline?

2 Answers2