13

I’m trying to add a service to a stack after the stack was already deployed. But this new service is having trouble communicating with services (redis) inside the stack.

This is my current understanding of stacks and services, please let me know if there are any inaccuracies.

Stacks are an abstraction on top of services that provide useful utilities like DNS so services across a stack can communicate with one another. Stacks allow us to logically separate out groups of services, that might be running on the same swarm (so different development teams can share the same swarm).

I would like to first deploy a stack to a swarm (via compose file) and then periodically add containers like the one described in this article about one-shot containers. These containers are different because they are performing long, stateful operations. They need to be spun up with some initial state, do their work, and then go away. They are different because they don’t need to be replicated, or load balanced.

Essentially what I’m trying to do is:

Launch a “stack” like this:

docker stack deploy --with-registry-auth --compose-file docker-compose.yml my-stack

And then some time later when certain criteria is met, add a container like this:

docker service create -name statefulservice reponame/imagename

And this generally behaves as expected, except statefulservice isn’t able to talk to redis inside my-stack.

I’m confident that statefulservice is engineered correctly because when it’s added to the docker-compose.yml it behaves as expected.

A further detail that may or may not be relevant is that the command to create a new service is issued from a container within the swarm. This happens using the go sdk for docker, and I’m using it the way the one-shot container article described

The reason I suspect this isn’t relevant: I still run into this issue when I do this operation via docker-cli only (and not use the docker sdk for go).

Parth Mehrotra
  • 2,712
  • 1
  • 23
  • 31
  • For clarity, most people should just update the stack file yaml and rerun `docker stack deploy` on it. It seems for some reason you don't want to do that, so the question title should read "how to create a separate Swarm service that can communicate with an existing stack". Technically that isn't adding a service to a stack because the new service is not in the stack file, and won't show up in stack commands. – Bret Fisher Jul 07 '18 at 23:01
  • 1
    Hi Bret, you make a good point, something I should have mentioned is that I use the `com.docker.stack.namespace` label upon creation to ensure that these stateful services are members of the same stack. So commands like `docker stack rm stackname` work as intended. – Parth Mehrotra Jul 13 '18 at 19:03

2 Answers2

15

When you deploy a stack like this:

docker stack deploy --with-registry-auth --compose-file docker-compose.yml my-stack

It creates a network called my-stack_default

So to launch a service that can communicate with services that are in that stack you need to launch them like this:

docker service create --name statefulservice --network my-stack_default reponame/imagename
Parth Mehrotra
  • 2,712
  • 1
  • 23
  • 31
  • 1
    Correct, sorry, I forgot you didn't want to just add services to the stack file, which is the typical way of adding services to a stack. Good luck! – Bret Fisher Jul 07 '18 at 22:58
  • Is it possible to add a new service to docker-compose and up just that service without any side effect on other services? – Rahman Jan 18 '21 at 14:19
2

It would help to have a working example, but:

My guess is you don't have the services on the same docker network. Even if you don't manually assign stack services to a network (which is fine), The stack command creates one where all services in that stack are attached to. You'll need to specify that overlay network in your subsequent service create commands so they can find each other in DNS and communicate inside the swarm.

For example, if I create a stack called nginx, it will add all those services (unless configured otherwise in stack file) to an overlay network called nginx_default

Bret Fisher
  • 8,164
  • 2
  • 31
  • 36