docker-compose push to ECR fails in codebuild

Question

I've been trying to figure this out for a couple days and the issue is compounded by the fact that I'm not getting a useful error message.

I'm using the following buildspec.yml file in codebuild to build docker containers and then send to AWS ECR.

version: 0.2
env:
  parameter-store:
    AWS_DEFAULT_REGION: "/docker_test/region"
    IMAGE_REPO_NAME: "/docker_test/repo_name"
    IMAGE_TAG: "/docker_test/img_tag"
    AWS_ACCOUNT_ID: "account_id"

phases:
  install:
    runtime-versions:
      docker: 18
  pre_build:
    commands:
      - echo Logging in to Amazon ECR and DockerHub...
      - docker login -u AWS -p $(aws ecr get-login-password --region $AWS_DEFAULT_REGION) $AWS_ACCOUNT_ID.dkr.ecr.$AWS_DEFAULT_REGION.amazonaws.com
  build:
    commands:
      - echo Build started on `date`
      - echo Building the Docker image...
      - docker-compose -f docker-compose.yml -f docker-compose.prod.yml build
  post_build:
    commands:
      - echo Build completed on `date`
      - echo Pushing the Docker image...
      - docker-compose -f docker-compose.yml -f docker-compose.prod.yml push
artifacts:
  files:
    - 'Dockerrun.aws.json'

I've tried docker 19, slightly different versions of the docker login line and made sure my roles were set. I get "login succeeded" so I assume the login line is good.

[Container] 2021/12/22 16:19:20 Running command docker login -u AWS -p $(aws ecr get-login-password --region $AWS_DEFAULT_REGION) $AWS_ACCOUNT_ID.dkr.ecr.$AWS_DEFAULT_REGION.amazonaws.com
WARNING! Using --password via the CLI is insecure. Use --password-stdin.
WARNING! Your password will be stored unencrypted in /root/.docker/config.json.
Configure a credential helper to remove this warning. See
https://docs.docker.com/engine/reference/commandline/login/#credentials-store

Login Succeeded

[Container] 2021/12/22 16:19:26 Phase complete: PRE_BUILD State: SUCCEEDED
[Container] 2021/12/22 16:19:26 Phase context status code:  Message: 
[Container] 2021/12/22 16:19:26 Entering phase BUILD

The post_build phase fails however with the following:

Successfully built d6878cbb68ba
Successfully tagged ***.dkr.ecr.***.amazonaws.com/***:latest

[Container] 2021/12/22 16:21:58 Phase complete: BUILD State: SUCCEEDED
[Container] 2021/12/22 16:21:58 Phase context status code:  Message: 
[Container] 2021/12/22 16:21:58 Entering phase POST_BUILD
[Container] 2021/12/22 16:21:58 Running command echo Build completed on `date`
Build completed on Wed Dec 22 16:21:58 UTC 2021

[Container] 2021/12/22 16:21:58 Running command echo Pushing the Docker image...
Pushing the Docker image...

[Container] 2021/12/22 16:21:58 Running command docker-compose -f docker-compose.yml -f docker-compose.prod.yml push
Pushing myapp (***.dkr.ecr.***.amazonaws.com/***:latest)...
The push refers to repository [***.dkr.ecr.***.amazonaws.com/***]
EOF

[Container] 2021/12/22 16:22:49 Command did not exit successfully docker-compose -f docker-compose.yml -f docker-compose.prod.yml push exit status 1
[Container] 2021/12/22 16:22:49 Phase complete: POST_BUILD State: FAILED
[Container] 2021/12/22 16:22:49 Phase context status code: COMMAND_EXECUTION_ERROR Message: Error while executing command: docker-compose -f docker-compose.yml -f docker-compose.prod.yml push. Reason: exit status 1
[Container] 2021/12/22 16:22:49 Phase complete: UPLOAD_ARTIFACTS State: SUCCEEDED
[Container] 2021/12/22 16:22:49 Phase context status code:  Message:

I'm just not sure how to get more information on this error - that would be ideal.

EDIT:

I'm adding the docker-compose.prod.yml file for additional context:

version: "3.2"

services:
  myapp:
    image: ${AWS_ACCOUNT_ID}.dkr.ecr.${AWS_DEFAULT_REGION}.amazonaws.com/${IMAGE_TAG}
    command: bash -c "
        python manage.py migrate
        && gunicorn --bind :8000 --workers 3 --threads 2 --timeout 240  project.wsgi:application"

    restart: always
    ports:
      - "80:80"

  celery_worker:
    image: ${AWS_ACCOUNT_ID}.dkr.ecr.${AWS_DEFAULT_REGION}.amazonaws.com/${IMAGE_TAG}
    command: celery -A project worker --loglevel=${CELERY_LOG_LEVEL:-WARNING}
    restart: always

A few things: does the repo exist in ECR? ECR does not (yet) support creating the repo on push and it needs to pre-exist. Does the role attached to that build project have a policy that can push to this ECR repo? Last, have you tried to do a `docker push` of the resulting image to see if the docker binary has a better error message that could put you on the right track? — mreferre, Dec 23 '21 at 07:58
@mreferre I'll give docker push a try probably, the role should be good that was one of the first things I suspected. The repo does exist I create it beforehand but that leads me to another question now - I'm wondering how I'm supposed to select the specific repo as currently I'm not, which is probably the issue... above I added my `docker-compose.prod.yml` for more context. — Marlone, Dec 31 '21 at 23:16

score 0 · Answer 1 · answered Jan 01 '22 at 00:45

OK, so I figured it out. Your question about making sure the repo exists pointed me in the right direction @mreferre. I was confused about the use of IMAGE_TAG and IMAGE_REPO_NAME in the code samples I referenced when trying to build this. They were essentially supposed to be the same thing so the push was failing because I was trying to push to an ECR repo named "proj-name" which didn't exist. I just needed to change it to "repo-name" so the image in docker-compose.prod.yml becomes:

image: ${AWS_ACCOUNT_ID}.dkr.ecr.${AWS_DEFAULT_REGION}.amazonaws.com/${IMAGE_REPO_NAME}

Glad you had it working now. Yes the repo name isn't the tag. The repo name represents the main name of your container image while the tag represents a date, a label.... anything that allows you to version multiple container images of the same application. If you push an image with the repo name only the registry will auto-assign the `LATEST` tag. — mreferre, Jan 01 '22 at 11:21

docker-compose push to ECR fails in codebuild

1 Answers1