Docker1.12 Worker not able to join in cluster(Swarm: Pending)

Question

Manager Version Docker version 1.12.0-rc5, build a3f2063,

Worker version Docker version 1.12.0-rc5, build a3f2063.

Created Swarm manger:

docker swarm init --advertise-addr "172.25.30.2:4243"

    Swarm initialized: current node (3kmewyb10p8xj3ke5rpjyw4s8) is now a manager.

    To add a worker to this swarm, run the following command:
        docker swarm join \
        --token SWMTKN-1-5lwzvv7au6hosiqqmdwmcxvmlmhtz4ts04jsg06284fq3posn0-enq26dqnwma38ij48hymtnioq \
        172.25.30.2:4243

    To add a manager to this swarm, run the following command:
        docker swarm join \
        --token SWMTKN-1-5lwzvv7au6hosiqqmdwmcxvmlmhtz4ts04jsg06284fq3posn0-85cwe5pf779qw0knjn6wxdbim \
        172.25.30.2:4243

Then created worker

docker swarm join --token SWMTKN-1-5lwzvv7au6hosiqqmdwmcxvmlmhtz4ts04jsg06284fq3posn0-enq26dqnwma38ij48hymtnioq 172.25.30.2:4243
    Error response from daemon: Timeout was reached before node was joined. Attempt to join the cluster will continue in the background. Use "docker info" command to see the current swarm status of your node.

I have checked logs in worker

time="2016-08-01T00:22:47.449844174-07:00" level=warning msg="failed to retrieve remote root CA certificate: rpc error: code = 1 desc = context canceled" 
time="2016-08-01T00:22:47.449962215-07:00" level=warning msg="failed to retrieve remote root CA certificate: rpc error: code = 1 desc = context canceled" 
time="2016-08-01T00:22:47.450025342-07:00" level=warning msg="failed to retrieve remote root CA certificate: rpc error: code = 1 desc = context canceled" 
time="2016-08-01T00:22:47.450081950-07:00" level=warning msg="failed to retrieve remote root CA certificate: rpc error: code = 1 desc = context canceled" 
time="2016-08-01T00:22:47.450142443-07:00" level=warning msg="failed to retrieve remote root CA certificate: rpc error: code = 1 desc = context canceled" 
time="2016-08-01T00:22:47.450202836-07:00" level=error msg="cluster exited with error: rpc error: code = 1 desc = context canceled" 
time="2016-08-01T00:23:31.351868722-07:00" level=error msg="Handler for POST /v1.24/swarm/join returned error: Timeout was reached before node was joined. Attempt to join the cluster will continue in the background. Use \"docker info\" command to see the current swarm status of your node."

In docker info, I saw "Swarm: Pending"

I did docker swarm update also!. Still, the worker was not able to join the cluster. So, how can I reslove

UPDATE-1

Uninstalled & removed config files and then install docker 1.12 again with version Docker version 1.12.0, build 8eab29e.

Still facing the same problem(Not able to join and "Swarm:Pending" in docker info) with DIFFERENT error in /var/logs/upstat/docker.logs

time="2016-08-01T11:22:08.629760770-07:00" level=error msg="Handler for POST /v1.24/swarm/join returned error: Timeout was reached before node was joined. Attempt to join the cluster will continue in the background. Use \"docker info\" command to see the current swarm status of your node."

Thanks.

Apparently, I had the same problem. The solution for me was a matter of fixing the date of the workers. To be clear, the date in the workers was not right (was an old date). I made the workers use a NTP server and that fixed the problem (I guess). I am not writing that as a solution because I don't fully understand what is going on. I just know that it worked for me. — Inti Gonzalez-Herrera, Aug 01 '16 at 15:23
Had the same issue. Probably because the generated token (to join a cluster) is dependent of the time. — lvthillo, Aug 09 '16 at 13:48
I had the same problem. @IntiGonzalez-Herrera 's mention of date/time was the solution. I was unable to setup the NTP server due to some firewall issues here but imagine I could at least use NTP to sync all of my workers to the manager. A quick solution I used for testing swarm was to sync the time with date --set="$(ssh user@server date)" — Chris Townsend, Sep 12 '16 at 16:51
i faced a same issue.servers are in different time zone.then came to same time zone fixed that issue. — dneranjan, Dec 20 '18 at 13:52

score 7 · Accepted Answer · edited Dec 14 '16 at 17:42

The thing is, I was trying to join with wrong "port" (As docker swarm init shown in output).

1) Before "docker swarm init", the docker running on port "4243" only. I have checked with netstat -tulp | grep docker. So I advertised with that port!

root@veeru:~# netstat -tulpn | grep docker
tcp6       0      0 :::4243                 :::*                    LISTEN      8750/dockerd 

root@veeru:~# docker swarm init --advertise-addr "172.25.30.2:4243"
Swarm initialized: current node (exvwgj0pu4cd124ljnblt9xff) is now a manager.

To add a worker to this swarm, run the following command:
    docker swarm join \
    --token SWMTKN-1-5j9mpo8hepue6g1sjdas33thr92w1o9hlef5auwqpbxs3glt39-6zomhgu204m9alq51f632nzas \
    172.25.30.2:4243

To add a manager to this swarm, run the following command:
    docker swarm join \
    --token SWMTKN-1-5j9mpo8hepue6g1sjdas33thr92w1o9hlef5auwqpbxs3glt39-axhgqgo4jqw4hv38x578m44wh \
    172.25.30.2:4243

2) After docker swarm init, the docker is running with 4 port including the port 2377(netstat -tupln | grep docker).

root@veeru:~# netstat -tulp | grep docker
tcp6       0      0 [::]:2377               [::]:*                  LISTEN      8750/dockerd    
tcp6       0      0 [::]:7946               [::]:*                  LISTEN      8750/dockerd    
tcp6       0      0 [::]:4243               [::]:*                  LISTEN      8750/dockerd    
udp6       0      0 [::]:7946               [::]:*                              8750/dockerd

In point 1, it is telling to run docker swarm join with port 4243 in worker. Previously I did run like that!.(It wont work!)

Later I did docker swarm leave and joined with port 2377. Now I am able to join!

docker swarm join --token SWMTKN-1-5j9mpo8hepue6g1sjdas33thr92w1o9hlef5auwqpbxs3glt39-6zomhgu204m9alq51f632nzas 172.25.30.2:2377

score 2 · Answer 2 · answered Feb 13 '18 at 09:40

For me it was a firewall issue too.

I tried to ping to the manager node and was pinging back
Checked if the ports are opening using telnet and was not able to connect and figured out it was the port issue.

If you are running Centos than the port can be easily opened using the firewalld

Check if the firewalld is running

sudo firewall-cmd --state

Opening the port you want

sudo firewall-cmd --zone=public --add-port=2377/tcp

Change the port as per your node ports it is trying to connect to.

score 2 · Answer 3 · edited Oct 02 '18 at 19:56

Just expose port 2377 of manager, it will work.

It clearly means node unable to connect manager, so timeout happening to conform same just do telnet manager-ip 2377 (don't try ping, won't work).

And if you are facing the same error even though all firewalls are disabled in both nodes and manager, then try to create another manager exposing port 2377 as below:

docker-machine create --driver amazonec2 --amazonec2-open-port 2377 manager1

And now try to join nodes to new manager created now, but port you are using to join should be 2377 if you gonna use diff then expose that port in above command. Doing same worked for me as I suspect it's because others used other different servers but I'm using same server for both manager and nodes.

score 2 · Answer 4 · answered Oct 08 '20 at 17:37

According to dockers website Here they stated the ports to enable. Run the following commands on both the Swam Manager and worker nodes

sudo ufw enable
sudo ufw allow 22/tcp
sudo ufw allow 2376/tcp
sudo ufw allow 2377/tcp
sudo ufw allow 7946/tcp
sudo ufw allow 7946/udp
sudo ufw allow 4789/udp
sudo ufw reload

We just gave access to the neccessary port. After running these commands, all docker commands should be working now.

score 1 · Answer 5 · answered Jun 08 '17 at 06:32

1

I was facing similar issue, While in my case port was getting blocked due to firewall rule.

answered Jun 08 '17 at 06:32

Vikrant Pawar

789
1
6
23

score 1 · Answer 6 · answered Jul 13 '17 at 09:56

I was having the same issue. I was running coreos vms in Azure. I found out that all my vms had the same private ip address and different public ip addresses. This usually happens when the vms are part of the same security group, however it was not the case this time. The issue was the my account had reached the max number of resources, so I deleted the resources such as ip addresses, nsg, networks etc and then re-provisioned new vms, they had different private ips and when ran the command everything was fine. My docker version is 1.12.6

score 0 · Answer 7 · answered Feb 14 '22 at 11:35

0

Assuming you did so; if you get "Connection time out" it means that there is a firewall preventing you from connecting. Either on the source host, or the destination host (e.g. iptables rules) or in between. If you are running on some public cloud, make sure that access lists (e.g. EC2 security groups) allow connections between hosts on that port

answered Feb 14 '22 at 11:35

KISHORE PRATHIPATI

1

It *could* be a firewall, or proxy configuration, or something else. – XouDo Feb 14 '22 at 13:59

score 0 · Answer 8 · answered Jul 02 '22 at 17:28

0

I was trying to connect 4 nodes(1 master, 3 slave) over EC2 ubuntu server ami image, For me it was an firewall issue. Check your security groups=>Inbound rules, for me it was custom, and I changed it to anywhere and it will work.

answered Jul 02 '22 at 17:28

Shubham Gupta

1

Docker1.12 Worker not able to join in cluster(Swarm: Pending)

8 Answers8

Linked