21

Basically, I need to configure CI/CD with bitbucket source code to ECS containers. I want to use CodePipline to deploy new ECR image to ECS.

Currently, there is no option in AWS CodePipline to specify bitbucket as the source. However, I've managed to configure CodeBuild with webhooks so it builds docker file and pushes it to ECR on every push to release branch.

I want to configure ECR as the "source" stage in CodePipline and deploy it to existing ECS cluster/service so deploy will be automated.

  • I specified "Amazon ECR" as action provider with "image_details" output artifact in source stage.
  • I specified "Amazon ECS" (not "Amazon ECS (Blue/Green)") as an action provider for deploy stage with "image_details" as input artifact

codepipeline_settings

Unfortunately, basic configuration and artifact chaining results if following error in the deploy step:

Invalid action configuration
The image definition file imageDetail.json contains invalid JSON format

Though "Amazon ECR" stage provides imageDetail.json as an output artifact, it does not seem to be expected for "Amazon ECS" deploy provider. Is there any rational way to get around this issue?

I'm aware, that it is possible to configure CI/CD with bitbucket + API Gateway/Lambda + CodePipeline, I also consider using CodeCommit instead of bitbucket as the source repo - still, hope there is a possible elegant solution to use bitbucket with CodePipeline directly.

UPD: I've ended up with pretty nice configuration, described in this blogpost: the overall idea is to allow CodeBuild to upload source code from bitbucket to S3 and then use CodePipeline with S3 as a source to deploy new docker image to ECR and publish new task definition revision in ECS cluster. S3 is still overhead and I'm searching for a more elegant solution for the task.

Sergey Nikitin
  • 845
  • 2
  • 13
  • 25
  • hi Sergey, Did you have a chance to find out why it's happening and we can't setup ECR as a source and deploy docker image to ECS? I have faced the same problem with `imageDetail.json`? – statut Apr 17 '19 at 06:17
  • 1
    @statut Hi! The original issue is still open. I decided to use another pipeline instead. The idea is described in https://lgallardo.com/2018/09/07/codepipeline-bitbucket. From my observations, imageDetail.json output of ECR cannot be used for deploy, so I would probably try using "Invoke" lambda stage to transform imageDetail.json into imagedefinitions.json of following format: [{"name":"","imageUri":""}] – Sergey Nikitin Apr 18 '19 at 07:17
  • thank you! The provided format helped me – statut Apr 19 '19 at 11:57
  • @statut I’ll appreciate if you provide some details on final solution so we can mark it as accepted answer. Would be helpful for the community. – Sergey Nikitin Apr 20 '19 at 12:06
  • The whole initial issue was CodePipeline not supporting bitbucket as the source provider. AWS has now added this feature: https://aws.amazon.com/blogs/devops/using-aws-codepipeline-and-aws-codestar-connections-to-deploy-from-bitbucket/ - yet, it has its limitations, so solutions discussed in this thread are still useful for some scenarios of commercial development. – Sergey Nikitin Jan 28 '21 at 15:28

2 Answers2

29

I had a similar use-case and hit the same problem. Bit of a long answer with a solution that addresses my use case...

  1. Use Case: Tools Account has the CodePipeline which does the CodeBuild and docker push to QA Accounts ECR. The image will have 2 tags - A "commit hash" and "latest". QA Account has a pipeline which executes when a new image is made available and this pipeline deploys the new image into the Fargate cluster in QA Account.
  2. Deployment Platform: Amazon ECS Standard Deployment Actions.
  3. Source: ECR.
  4. Cause of the Problem: ECR Source's Output artifact rightfully only contains the Information about the Image and does not include the Container Name which is expected by ECS Standard Deploy.
  5. Suggestion to AWS: Given the popular scenario of using ECR as a source for ECS we can make an argument of having capability (optional) to add Container name in the output artifact - This would allow generation of Output artifacts which are acceptable as input for ECS Standard Deploy.

As per this official doco from AWS ECS Standard Deployment expects a - imagedefinitions.json file which provides the container name and image URI. It should looks like:

[
  {
    "name": "sample-app",
    "imageUri": "11111EXAMPLE.dkr.ecr.us-west-2.amazonaws.com/ecs-repo:latest"
  }
]

But the ECR source produces an Output Artifact called imageDetail.json example below. This does not match the expected input format for ECS Standard Deploy aka imagedefinitions.json - which includes the Container Name (name) and the deploy fails with a message like Deploy Failure:

{
    "ImageSizeInBytes": "44728918",
    "ImageDigest": "sha256:EXAMPLE11223344556677889900bfea42ea2d3b8a1ee8329ba7e68694950afd3",
    "Version": "1.0",
    "ImagePushedAt": "Mon Jan 21 20:04:00 UTC 2019",
    "RegistryId": "EXAMPLE12233",
    "RepositoryName": "dk-image-repo",
    "ImageURI": "ACCOUNTID.dkr.ecr.us-west-2.amazonaws.com/dk-image-repo@sha256:example3",
    "ImageTags": [
        "latest"
    ]
}

The approach I took to fix this is:

  1. In the Source stage: In addition to ECR source I added a source from s3 which contains imagedefinitions.json in a zip.

  2. In the ECS Deploy stage action I refer to the Output artifact from the s3 source which contains imagedefinitions.json in the format that ECS Standard Deploy understands.

Note: The imagedefinitions.json is static in the s3 bucket and always refers to latest tag on the said image. So in the QA image definitions bucket I will end-up with an image definitions zip i.e one per instance of Fargate service.

I've exported my pipeline here for general reference:

{
"pipeline": {
    "roleArn": "arn:aws:iam::ACCOUNTID:role/service-role/AWSCodePipelineServiceRole-REGION-PIPELINENAME",
    "stages": [
        {
            "name": "Source",
            "actions": [
                {
                    "inputArtifacts": [],
                    "name": "Source",
                    "region": "REGION",
                    "actionTypeId": {
                        "category": "Source",
                        "owner": "AWS",
                        "version": "1",
                        "provider": "ECR"
                    },
                    "outputArtifacts": [
                        {
                            "name": "SourceArtifact"
                        }
                    ],
                    "configuration": {
                        "ImageTag": "latest",
                        "RepositoryName": "PIPELINENAME"
                    },
                    "runOrder": 1
                },
                {
                    "inputArtifacts": [],
                    "name": "sourceimagedeffile",
                    "region": "REGION",
                    "actionTypeId": {
                        "category": "Source",
                        "owner": "AWS",
                        "version": "1",
                        "provider": "S3"
                    },
                    "outputArtifacts": [
                        {
                            "name": "PIPELINENAME-imagedefjson"
                        }
                    ],
                    "configuration": {
                        "S3Bucket": "BUCKETNAME",
                        "PollForSourceChanges": "true",
                        "S3ObjectKey": "PIPELINENAME.zip"
                    },
                    "runOrder": 1
                }
            ]
        },
        {
            "name": "Deploy",
            "actions": [
                {
                    "inputArtifacts": [
                        {
                            "name": "PIPELINENAME-imagedefjson"
                        }
                    ],
                    "name": "Deploy",
                    "region": "REGION",
                    "actionTypeId": {
                        "category": "Deploy",
                        "owner": "AWS",
                        "version": "1",
                        "provider": "ECS"
                    },
                    "outputArtifacts": [],
                    "configuration": {
                        "ClusterName": "FARGATECLUSTERNAME",
                        "ServiceName": "PIPELINENAME",
                        "FileName": "imageDetail.json"
                    },
                    "runOrder": 1
                }
            ]
        }
    ],
    "artifactStore": {
        "type": "S3",
        "location": "codepipeline-REGION-555869339681"
    },
    "name": "PIPELINENAME"
}
kiran01bm
  • 682
  • 7
  • 18
  • Have you also posted this suggestion to AWS on support or ECR/CodePipeline git? It looks like not that big feature to implement. – Sergey Nikitin Aug 06 '19 at 07:10
  • 1
    @SergeyNikitin - I've posted this on the ECR forum. [Link here](https://forums.aws.amazon.com/thread.jspa?threadID=307875) if you would like to follow. – kiran01bm Aug 13 '19 at 07:23
  • 1
    Thank you, I ran into this exact thing. Static S3 zip did the trick! – clay Jun 04 '20 at 19:24
  • 1
    Just got this working with static S3, thanks! In your exported pipeline I changed "FileName": "imageDetail.json" to "FileName": "imagedefinitions.json" and my static S3 file contains an imagedefinitions.json file - as this is more clear it's acting on the imagedef file. – Wheat Jul 06 '20 at 15:21
25

I just recently had to solve a similar issue where I wanted to use ECR as the source of my pipeline and have it deploy the image to ECS. The solution I found was by creating 3 stages:

  • Source: ECR
  • Build: Custom code to convert the ECR artifact into an artifact that the Deploy stage would understand
  • Deploy: To ECS

Here's the buildspec.yml file I'm using as my build stage:

version: 0.2

phases:
  install:
    runtime-versions:
      python: 3.7
  build:
    commands:
      - PHP_REPOSITORY_URI=$(cat imageDetail.json | python -c "import sys, json; print(json.load(sys.stdin)['ImageURI'].split('@')[0])")
      - IMAGE_TAG=$(cat imageDetail.json | python -c "import sys, json; print(json.load(sys.stdin)['ImageTags'][0])")
      - echo $PHP_REPOSITORY_URI:$IMAGE_TAG
  post_build:
    commands:
      - echo Writing image definitions file...
      - printf '[{"name":"container","imageUri":"%s"}]' $PHP_REPOSITORY_URI:$IMAGE_TAG > imagedefinitions.json
artifacts:
    files: imagedefinitions.json

Basically what this does is read the imageDetail.json file and extract the ECR repository URL and TAG and output a json file formatted for the ECS Deploy stage, which is just a standard stage without customization.

Carlos Barros
  • 266
  • 4
  • 4
  • I think this is it, going to test it and accept an answer shortly! – Sergey Nikitin Jul 13 '19 at 08:20
  • It took time to get back to this, but I've finally managed to create a working pipeline using your script. Thanks! – Sergey Nikitin Jul 22 '19 at 06:21
  • I am facing a similar issue, Can you share sample pipeline export output – Ragav Oct 04 '19 at 10:12
  • Awesome. Could you please share imagedefinitions.json file for our reference?? - @carlos-barros – Pushparaj Feb 10 '20 at 08:23
  • @Pushparaj the imagedefinitions.json file is generated by the buildspec.yml script above, and only contains the following: `[{"name":"container","imageUri":"123456.dkr.ecr.us-east-1.amazonaws.com/some/image:tag"}]` the imageUri is dynamically generated based on the repository uri and image tag read from imageDetail.json – Carlos Barros Feb 11 '20 at 16:54
  • This is nice! Got it working after a few changes: From CodePipeline, I added an environment variable CONTAINER_NAME=xxxx that matches the task-definition container name for the selected service on ECS. Then on the script, use it instead of hardcoded "container" string. – Juan Carrey Jun 24 '20 at 09:08
  • @CarlosBarros Thanks! This worked for me. I have another problem which is similar to this. Instead of ECS as a deploy destination, I want to use Lambda with ECR. Can you please help me out? https://stackoverflow.com/questions/69008142/aws-lambda-codedeploy-using-ecr-image-as-a-source – Abhishek Sep 01 '21 at 07:47
  • sorry to ask may be naive question.. I am not able to find option to add custom code in build action from console. can you help with that? – Dhruv Jan 10 '23 at 09:47
  • @Dhruv the custom code is defined in the buildspec.yml file. It reads the imageDetail.json file and extract the image url and tag and finally creates the imagedefinitions.json file used by the deploy stage. You can see it commands in the original post, inside the build and post_build steps. – Carlos Barros Jan 19 '23 at 02:38