4

I've been trying to find an efficient way to handle continuous deployment with a Docker compose setup and AWS hosting.

So far I've looked into CodeDeploy, S3 buckets, and ECS. My application is relatively small with only 3 docker services, a Django app, NGINX, and PostgreSQL. I was unable to find any reliable information for using CodeDeploy with Docker compose and because of the small scale ECS seems impractical. I've considered an S3 bucket but that seems no better than just deploying my application with something like git or scp.

What is a standard way of handling deploying a docker compose setup on AWS? If possible I would like to use Bitbucket Pipelines or CircleCI to perform the deployment in a manually triggered step after running tests. But I've been unable to find a solution that would easily let me copy over the code (which is in a git repo on a production branch and is how I get the code onto the production server at the moment).

  • The standard way of copying over the code is to `docker build` it into an image and push it into a repository (in an AWS context, perhaps ECR). – David Maze Dec 02 '18 at 16:47
  • This posterior question might help you: https://stackoverflow.com/questions/61122181/terraform-deploying-a-docker-compose-app-on-eks-ecs – Johann8 Apr 17 '20 at 09:48

3 Answers3

0

What I will do in your case is:

1 - If needed, update your docker-compose.yml file (or however you called it) to version 3 or higher, to use swarm.

2 - During your pipeline build all images needed, and push them to a registry.

3 - In your pipeline scp your compose file to a manager node.

4 - Deploy your application using swarm (docker stack deploy -c <your-docker-compose-file> your_app_name). This way you can handle rolling updates and scale easily.

Note that if you want to use multiple nodes you need to open a few ports in them

gasc
  • 638
  • 7
  • 14
  • Using swarm seems overboard for a small app like this. Can I achieve the same thing without it? –  Dec 02 '18 at 18:48
0

I would like to add some possibilities to @gasc answer

  1. It would be better if you make a cloudformation template for deploying your EC2 resources with all required groups, auto scaling and other stuff.

  2. Then Create the AMI with docker compose installed or any other thing you would be required for your ec2 enviroment.

  3. Then you can use code deploy pipeline, here also aws provides private container registry may be you want to use that

  4. Rest of the steps are same just SCP the compose file into EC2 launch docker-compose up command and you are done.

Let me know if you want more help I'm open for discussion

varnit
  • 1,859
  • 10
  • 21
  • 1
    This is exactly the solution I'm looking for. However I've attempted to implement it and had trouble figuring out how to setup the templates and the appspec.yml file. Could you possibly provide some examples of setting up a template, the appspec for codedeploy, and setting up compose on an AMI ? –  Dec 02 '18 at 18:49
  • please be more specific in appspec.yml you need to just launch a cloudfomation template and use commands to build your new docker images and then copy your code files and docker-compose.yml to your ec2, you also need to create your AMI the things are simple enough to understand from aws offiicial docs but if you need help about something specific let me know – varnit Dec 03 '18 at 15:56
0

I see you mentioned that ECS might seem impractical for such a small scale - in my opinion not necesarilly. It would require of you to rewrite your docker-compose.yml into task and services definitions, but since there's not a lot of services, that shouldn't take you much time.

pgrzesik
  • 1,869
  • 13
  • 14
  • I'd like to give it a try. Could you provide a simple example on how I would need to change my compose file? –  Dec 03 '18 at 00:26
  • 1
    It turns out that `ecs-cli` directly supports compose files! https://docs.aws.amazon.com/AmazonECS/latest/developerguide/cmd-ecs-cli-compose-parameters.html I personally did not use that approach, and I'm using Task and Service definitions via CloudFormation, similar to approach presented in this example: https://github.com/aws-samples/ecs-refarch-cloudformation/blob/master/services/product-service/service.yaml. Task definition corresponds to a single service definition from compose file and most of the parameters are intuitive to map. – pgrzesik Dec 03 '18 at 23:50
  • @Progressive So I just need to write a task definition YAML file ? I'm looking at the docs and I'm a little confused on how to write the task definition config file. I just want to run docker compose on a host and preferably be able to automate deployment to it from CircleCI and/or bitbucket pipelines. Also it seems that ecs-cli just uses the docker compose syntax for their own task management and doesn't actually support docker compose. –  Dec 04 '18 at 02:36
  • You can use your docker-compose files to run tasks on ECS with ecs-cli, here's a bit of the reference from the docs https://docs.aws.amazon.com/AmazonECS/latest/developerguide/cmd-ecs-cli-compose-start.html (I know the docs of AWS can sometimes be, let's say, not good). – pgrzesik Dec 04 '18 at 21:57
  • If you want to skip using compose files, then yes, you have to write yaml for task definition, create a task definition from this yaml specification (using for example https://docs.aws.amazon.com/cli/latest/reference/ecs/register-task-definition.html) and later, you can actually run your task with https://docs.aws.amazon.com/cli/latest/reference/ecs/run-task.html. – pgrzesik Dec 04 '18 at 21:57