6

I have a few basic questions on scaling Docker containers:

I have 5 different apps. They are not connected to each other. Before having containers I would run 1 app per VM and scale them up and down individually in the cloud.

Now with containers I get the isolation on top of a VM, so now I can potentially run one host with 5 docker containers where each app is isolated in its own container.

As long as I have enough resources on my host I can scale up and down those containers individually as my traffic grows or shrinks. e.g. I have 3 containers running app 1, but only 1 container running app 2.

At peak times app 3 gets a lot of traffic and I need to launch a 2nd host which runs only containers for app 3.

My first question is if the above makes sense what I say or if I have misunderstood something. My second question is what technology is currently available to get this all done in an automated way. I need a load balancer and an auto scaling group which is capable of the above scenario without me having to do manual interventions.

I looked into AWS ECS and am not quite sure if it can satisfy my needs as I outlined it above.

Does anyone know how to achieve this, or is there a better way of managing and scaling my 5 apps which I am missing?

UPDATE:

Via Twitter I have been pointed to Kubernetes and specifically to the docs on the Horizontal Pod Autoscaler.

Might be useful for others as well. I will update this question as I learn more.

dustinmoris
  • 3,195
  • 3
  • 25
  • 33
  • 3
    It should be impossible on StackOverflow to downvote a question without providing a comment on why -.- – dustinmoris Jan 05 '16 at 14:36
  • I'm guessing you were downvoted because this doesn't appear to be a question about programming, so it is off topic for this site. I think it would be more appropriate on serverfault.com. – Mark B Jan 05 '16 at 15:24
  • 1
    Okay, perhaps you are right, but where is the line today when it comes to DevOps questions? Is it more Dev or more Ops? Also this question is similar and hasn't been downvoted: http://stackoverflow.com/questions/18285212/how-to-scale-docker-containers-in-production – dustinmoris Jan 05 '16 at 17:05
  • 1
    It is an increasingly blurry line. And I'm just telling you why you might be getting downvotes. I can't tell you why the other question didn't get downvotes, it appears to be a better fit for serverfault as well. I'm not trying to justify the downvotes, or justify the separation of the stackexchange sites, I'm just letting you know why I see downvotes happen often on here. I personally feel that software engineers with their inherent need to separate and codify things have split the stackexchange network into too many sites with overlapping topics. – Mark B Jan 05 '16 at 17:24
  • makes sense what you say, thanks for the feedback and I will certainly post it there as well! – dustinmoris Jan 05 '16 at 17:26
  • 2
    I would suggest making some noise here: https://meta.stackoverflow.com/questions/271279/topicality-of-devops-questions My DevOps questions are always getting downvoted as well no matter if I post it on StackOverflow Or ServerFault and on PowerUser nobody responds. – Jan Vladimir Mostert Jan 05 '16 at 18:01
  • @dustinmoris Did you consider using Google Cloud Container Engine? It will do exactly what you are asking for. You can find more information here: https://cloud.google.com/container-engine/ – George Jan 06 '16 at 20:14
  • Good question in my mind, on a related note I've asked something similar however with a focus on scaling from 0 hosts & containers upwards and then back to 0 to avoid idle time costs: http://stackoverflow.com/questions/34599589/application-startup-and-shutdown-based-on-authenticated-user-activity – Nico de Wet Jan 06 '16 at 22:15
  • @dustinmoris what did you end up going with? I updated my answer - I think you can achieve this using AWS ECS. – poida Mar 02 '17 at 09:52
  • 1
    @poida We ended up using both, Kubernetes and AWS ECS. At the time of writing this question AWS ECS was different that it is today, especially the new application load balancer has made a huge difference. We have two different teams and one uses Kubernetes and the other AWS ECS now. I think both seem pretty happy. – dustinmoris Mar 02 '17 at 14:18

3 Answers3

5

There are several options, but none that I know that does it all: you will need 2 things: autoscaling hosts according to signals, then autoscale containers on the hosts.

The following are the solutions to deploy and scale containers on the hosts (not necessarily auto-scale though):

Kubernetes is an orchestration tool which allows to schedule and (with the optional autoscaler) to autoscale pods (groups of containers) in the cluster. It makes sure your containers are running somewhere if a host fails. Google Container Engine (GKE) offers this as a service, however i am not sure they have the same functionalities to autoscale the number of VMs in the cluster as AWS does.

Mesos: somewhat similar to Kubernetes but not dedicated to running containers.

Docker Swarm: the Docker multi-host deployment solution, allows you control many hosts as if they were a single Docker host. I don't believe it has any kind of 'autoscaling' capability, and I don't believe it takes care of making sure pods are always running somewhere: it's basically docker for cluster.

[EDIT] Docker supports restarting failing containers with the restart=always option, also, as of Docker 1.11 Docker Swarm is a mode in Docker Daemon, and supports rescheduling containers on node failure: it will restart containers on a different node if a node is no longer available.

Docker 1.11+ is becoming a lot like Kubernetes in terms of functionalities. It has some nice features (like TLS between nodes by default), but still lacks things like static IPs and storage provisioning

None of these solutions will autoscale the number of hosts for you, but they can scale the number of containers on the hosts.

For autoscaling hosts, solutions are specific to your cloud provider, so these are dedicated solution. The key part for you is to integrate the two: AWS allows deployment of Kubernetes on CoreOS; I don't think they offer this as a service, so you need to deploy your own CoreOS cluster and Kubernetes.

Now my personal opinion (and disclaimer)

I have mostly used Kubernetes on GKE and bare-metal, as well as Swarm a about 6 months ago, and i run an infra with ~35 services on GKE:

Frankly, GKE with Kubernetes as a Service offers most of what you want, but it's not AWS. Scaling hosts is still a bit tricky and will require some work.

Setting up your own Kubernetes or Mesos on AWS or bare metal is very feasible, but there is quite a learning curve: it all depends if you really strongly feel about being on AWS and are willing to spend the time.

Swarm is probably the easiest to get working with, but more limited, however homebuilt script can well do the job core job: use AWS APIs to scale hosts, and Swarm to deploy. The availability guarantee though would require you monitoring and take care of re-launching containers if a node fails.

Other than that, there are also container hosting providers that may do the job for you:

MrE
  • 19,584
  • 12
  • 87
  • 105
  • Using cloud provider APIs to scale hosts seems like the way to go - but naturally one would not scale hosts independent of underlying container scaling requirements (up or down). It seems a controller is required that manages host and container level scaling with the latter triggering the former (up and down). – Nico de Wet Jan 06 '16 at 22:22
0

I would take a look at Tutum(Recently acquired by Docker actually). It ties into CI and I believe it has autoscaling capabilities.

https://www.tutum.co/

errata
  • 23,596
  • 2
  • 22
  • 32
0

UPDATE: this is supported by AWS ECS with Task Placement Constraints.

  1. Have your ECS cluster served by two auto-scaling groups (ASGs).

  2. In the first ASG, set min, max and desired size all to 1.

    Tag this instance with a custom attribute ALLOW_ALL_APPS = TRUE. Do this in the user data script.

  3. In the second ASG, set min and desired size to 0 and max size to 1 (I'm assuming you only want 2 instances).

    Tag this instance with custom attribute ALLOW_ALL_APPS = FALSE. Again in the user data script.

  4. The scale up alarm for the second ASG will be determined by load on the first ASG.

    If you know when peak times are for app 3 you could bump it up preemptively with a scheduled scaling action.

  5. Scaling down for the second ASG is when its load drops enough so that the first ASG can handle it on its own.

  6. In the service definitions for apps 1, 2, 4 and 5 you would have placement constraints restricting them to run only on nodes where ALLOW_ALL_APPS = TRUE.

  7. In the service definition for app 3 there are no placement constraints.

  8. Service auto scaling is configured for all apps based on container or application metrics.

Community
  • 1
  • 1
poida
  • 3,403
  • 26
  • 26