35

I'm working on a project to set up a cloud architecture using docker-swarm. I know that with swarm I could deploy replicas of a service which means multiple containers of that image will be running to serve requests.

I also read that docker has an internal load balancer that manages this request distribution.

However, I need help in understanding the following:

Say I have a container that exposes a service as a REST API or say its a web app. And If I have multiple containers (replicas) deployed in the swarm and I have other containers (running some apps) that talk to this HTTP/REST service.

Then, when I write those apps which IP:PORT combination do I use? Is it any of the worker node IP's running these services? Will doing so take care of distributing the load appropriately even amongst other workers/manager running the same service?

Or should I call the manager which in turn takes care of routing appropriately (even if the manager node does not have a container running this specific service)?

Thanks.

user239558
  • 6,964
  • 1
  • 28
  • 35
Shabirmean
  • 2,341
  • 4
  • 21
  • 34

2 Answers2

37

when I write those apps which IP:PORT combination do I use? Is it any of the worker node IP's running these services?

You can use any node that is participating in the swarm, even if there is no replica of the service in question exists on that node. So you will use Node:HostPort combination. The ingress routing mesh will route the request to an active container.

One Picture Worth Ten Thousand Words

enter image description here

Will doing so take care of distributing the load appropriately even amongst other workers/manager running the same service?

The ingress controller will do round robin by default.

Now The clients should use dns round robin to access the service on the docker swarm nodes. The classic DNS cache problem will occur. To avoid that we can use external load balancer like HAproxy.

enter image description here

Farhad Farahi
  • 35,528
  • 7
  • 73
  • 70
  • 2
    Are you sure it always does round-robin? Can you link to that fact? It strikes me as an odd design choice. Could generate a lot of needless network traffic, e.g. if one container needs the service of another, and there's a replica on the same host - that replica should be given priority, to avoid needless network traffic. – Assaf Lavie Feb 28 '17 at 15:44
  • @farhad: Thank You. I see that the load-balancing mess works between the nodes in the swarm. But let's say I have isolated containers to different overlay networks set across nodes. Does the load balancing still happen? Ex: Node++ --> Container(A)_1, Container(A)_2, ContainerB(1) Node** --> Container(C)_1, Container(D)_1, Container(E)_1 Both services of type Container(A) are connected to three different overlay-network each of which only has C, D & E that run on Node**. So when C,D or E tries to access containers of type A over their own overlay network will it be handled? – Shabirmean Feb 28 '17 at 15:52
  • 1
    @AssafLavie Container to container traffic uses dns. Swarm internal DNS will return all active container records matching the service name in roundrobin order. Kubernetes handle this issue via pod concept. – Farhad Farahi Feb 28 '17 at 16:18
  • @Shabirmean When C,D,E try to access container A, they will query service A from dns, and DNS is based on the network they joined. In this Case the returned IP will be in overlay 1. – Farhad Farahi Feb 28 '17 at 16:21
  • @FarhadFarahi: Thanks Again. So when C wants to access A, it will query its internal DNS which will return all containers of type A in the same overlay. Let's say the DNS query returns **IP_A1** and **IP_A2** (both being in the same overlay as C), who does the load balancing now? Because I thought the _ingress mess_ works at the node level and not container level. – Shabirmean Feb 28 '17 at 16:42
  • @Shabirmean ips will be returned via dns round robin, So the dns will order the ips differently each time. Ingress load balancer will be used at node level only, when the packets are coming from nodes external ip address. – Farhad Farahi Feb 28 '17 at 16:49
  • @FarhadFarahi thanks. So, giving priority to same-host containers of a service is up to the user, then? (i.e. resolving dns and choosing same IP address if available?) – Assaf Lavie Feb 28 '17 at 16:56
  • @AssafLavie No, It works like a normal dns, Not aware of container placement. Your idea indeed improves network performance, but is not distributing load fair, and may overload a container. In certain situation, when containers are in pairs, Like kubernetes pods it will actually improves network performance because the whole pod traffic is getting managed via service (loadbalancer) – Farhad Farahi Feb 28 '17 at 17:00
  • yea, pods are really useful. I had in mind e.g. a web server that needs access to a redis cache. So, for stateless containers, the load balancing concern is less relevant.. – Assaf Lavie Feb 28 '17 at 17:03
  • @AssafLavie Thats right, even storage wise, for example log analysis, Nginx + logstash form nice pods with shared volumes. Although if you are using ceph it become irrelevant. – Farhad Farahi Feb 28 '17 at 17:06
  • @FarhadFarahi: So, even if I am not routing through the node's external IP, but using the overlay network to communicate to a specific service. Will the load be distributed evenly across all containers of the same service type that reside in the same overlay?? – Shabirmean Feb 28 '17 at 17:15
  • @Shabirmean via dns roundrobin, not ingress controller. And its not absolute roundrobin, because some applications cache the dns response for sometime, and do not honor ttl. – Farhad Farahi Feb 28 '17 at 17:27
  • @FarhadFarahi So this does not guarantee complete load-balancing if we try to distribute load across overlay networks that span across multiple nodes. We will have to implement our own "balancer" service per overlay (if DNS round robin is not sufficient) or architecture the deployment such that these inter-container calls are through actual swarm nodes. Can you please confirm my understanding one last time? Is there a docker community forum where I can ask the same question? – Shabirmean Feb 28 '17 at 17:36
  • @Shabirmean for outside clients ingress will do absolute loadbalancing, for inter container, Most of the time dns round robin will do a good job, and you can always fix your application. I recommend reading about kubernetes services, its exactly what you need :). There is a docker forum as well as docker repository on github. – Farhad Farahi Feb 28 '17 at 17:44
  • @FarhadFarahi Thank You Very Much :) – Shabirmean Feb 28 '17 at 17:49
  • Thanks for this. Why is HAProxy suggested over the built-in ingress load balancer? – NeilS May 01 '17 at 10:34
  • 1
    @NeilS Its not over ingress, its over DNS roundrobin! because clients may connect to the failed docker host, they have to retry another dns entry, but with HAproxy high availabled via VRRP (keepalived) there is no delay in case of failed host. – Farhad Farahi May 01 '17 at 14:31
3

An important additional information to the existing answer

The advantage of using a proxy (HAProxy) in-front of docker swarm is, swarm nodes can reside on a private network that is accessible to the proxy server, but that is not publicly accessible. This will make your cluster secure.

If you are using AWS VPC, you can create a private subnet and place your swarm nodes inside the private subnet and place the proxy server in public subnet which can forward the traffic to the swarm nodes.

When you access the HAProxy load balancer, it forwards requests to nodes in the swarm. The swarm routing mesh routes the request to an active task. If, for any reason the swarm scheduler dispatches tasks to different nodes, you don’t need to reconfigure the load balancer.

For more details please read https://docs.docker.com/engine/swarm/ingress/

Muhammad Tariq
  • 3,318
  • 5
  • 38
  • 42