2

I know k3d can do this magically via k3d cluster create myname --token MYTOKEN --agents 1, but I am trying to figure out how to do the most simple version of that 'manually'. I want to create a server something like:

 docker run -e K3S_TOKEN=MYTOKEN rancher/k3s:latest server

And connect an agent something like like:

 docker run -e K3S_TOKEN=MYTOKEN -e K3S_URL=https://localhost:6443 rancher/k3s:latest agent

Does anyone know what ports need to be forwarded here? How can I set this up? Nearly everything I try, the agent complains about port 6444 already in use, even if I disable as much as possible about the server with any combination of --no-deploy servicelb --disable-agent --no-deploy traefik

Feel free to disable literally everything other than the server and the agent, I'm trying to make this ultra ultra simple, but just butting my head against a wall at the moment. Thanks!

ricky116
  • 744
  • 8
  • 21

1 Answers1

2

The containers must "see" each other. Docker isolates the networks by default, so "localhost" in your agent container is the agent container itself.

Possible solutions: Run both containers without network isolation using --net=host, map API port of the server to the host with --port and use the host IP in the agent container or use docker-compose.

A working example for docker-compose is described here: https://www.trion.de/news/2019/08/28/kubernetes-in-docker-mit-k3s.html

Thomas
  • 11,272
  • 2
  • 24
  • 40
  • Thanks for your answer! Using the example .yaml, I can start the cluster. But I'm trying to replicate what the docker-compose does with individual containers via `docker run`. For example, I can start the server with docker-compose, but then if I want to add an agent (via `docker run --tmpfs /run --tmpfs /var/run --privileged -e K3S_CLUSTER_SECRET=somethingtotallyrandom -e K3S_URL=https://server:6443 rancher/k3s:latest` which matches the worker config in the .yaml), it is unable to connect due to `failed to get CA certs: Get \"https://127.0.0.1:6444/cacerts\`. Any thoughts? – ricky116 Jul 20 '21 at 07:51
  • 1
    Oh, I think I've got it! The docker-compose was implicitly starting a network for the containers defined within it. If I do `docker network create NAME` and then in my `docker run` commands I include the `--network NAME` parameter, they are now able to find eachother based on hostnames. So I can start as many agents as I like within this network! So I think I've solved it. Thanks for the link, that docker-compose.yaml was crucial – ricky116 Jul 20 '21 at 08:42
  • Glad, that it helped, please accept the answer in that case. – Thomas Jul 20 '21 at 09:05
  • @ricky116, can you please post your full invocation of both the server and the agent (master and worker)? – Mihai Galos Feb 19 '23 at 21:34
  • 1
    Here's mine: `sudo docker run --rm -it --net=k3s-home --privileged -e K3S_CLUSTER_SECRET=somethingtotallyrandom --hostname master rancher/k3s:latest server` and then `sudo docker run --rm -it --net=k3s-home --tmpfs /run --tmpfs /var/run --privileged -e K3S_TOKEN=somethingtotallyrandom -e K3S_URL=https://master:6443 --hostname worker rancher/k3s:latest agent --token `. I've got the by doing a `docker exec -it sh` and then `/var/lib/rancher/k3s/server/node-token`. I'm pretty sure this can be automated. – Mihai Galos Feb 19 '23 at 21:46