1

I received the error below:

{"t":{"$date":"2022-12-24T11:00:54.895+00:00"},"s":"I",  "c":"NETWORK",  "id":4712102, "ctx":"ReplicaSetMonitor-TaskExecutor","msg":"Host failed in replica set","attr":{"replicaSet":"{Replset_name}","host":"{VPS_IP}:27019","error":{"code":18,"codeName":"AuthenticationFailed","errmsg":"Authentication failed."},"action":{"dropConnections":false,"requestImmediateCheck

That error is shown in mongod.log when I add Secondary in the existing replica set MongoDB between different docker containers in 2 server machines.

My replica set structure includes the following:

  • Primary on VPS1_DO:container1 (actived) (same overlay-network)
  • Secondary1 on VPS1_DO:container2 (actived) (same overlay-network)
  • Secondary2 on VPS2_Azure:container1 (error)

Details in rs.status()

"members" : [
                {
                        "_id" : 0,
                        "name" : "10.5.0.11(staticIP-container1):27017",
                        "health" : 1,
                        "state" : 2,
                        "stateStr" : "SECONDARY",
                        "uptime" : 258904,
                        "optime" : {
                                "ts" : Timestamp(1672026076, 1),
                                "t" : NumberLong(67)
                        },
                        "optimeDurable" : {
                                "ts" : Timestamp(1672026076, 1),
                                "t" : NumberLong(67)
                        },
                        "optimeDate" : ISODate("2022-12-26T03:41:16Z"),
                        "optimeDurableDate" : ISODate("2022-12-26T03:41:16Z"),
                        "lastAppliedWallTime" : ISODate("2022-12-26T03:41:16.739Z"),
                        "lastDurableWallTime" : ISODate("2022-12-26T03:41:16.739Z"),
                        "lastHeartbeat" : ISODate("2022-12-26T03:41:17.962Z"),
                        "lastHeartbeatRecv" : ISODate("2022-12-26T03:41:18.521Z"),
                        "pingMs" : NumberLong(0),
                        "lastHeartbeatMessage" : "",
                        "syncSourceHost" : "10.5.0.12:27017",
                        "syncSourceId" : 1,
                        "infoMessage" : "",
                        "configVersion" : 17,
                        "configTerm" : 67
                },
                {
                        "_id" : 1,
                        "name" : "10.5.0.12(staticIP-container2):27017",
                        "health" : 1,
                        "state" : 1,
                        "stateStr" : "PRIMARY",
                        "uptime" : 589529,
                        "optime" : {
                                "ts" : Timestamp(1672026076, 1),
                                "t" : NumberLong(67)
                        },
                        "optimeDate" : ISODate("2022-12-26T03:41:16Z"),
                        "lastAppliedWallTime" : ISODate("2022-12-26T03:41:16.739Z"),
                        "lastDurableWallTime" : ISODate("2022-12-26T03:41:16.739Z"),
                        "syncSourceHost" : "",
                        "syncSourceId" : -1,
                        "infoMessage" : "",
                        "electionTime" : Timestamp(1671767185, 1),
                        "electionDate" : ISODate("2022-12-23T03:46:25Z"),
                        "configVersion" : 17,
                        "configTerm" : 67,
                        "self" : true,
                        "lastHeartbeatMessage" : ""
                },
                {
                        "_id" : 2,
                        "name" : "178.128.xx.xxx(IP-VPS):27019",
                        "health" : 0,
                        "state" : 6,
                        "stateStr" : "(not reachable/healthy)",
                        "uptime" : 0,
                        "optime" : {
                                "ts" : Timestamp(0, 0),
                                "t" : NumberLong(-1)
                        },
                        "optimeDurable" : {
                                "ts" : Timestamp(0, 0),
                                "t" : NumberLong(-1)
                        },
                        "optimeDate" : ISODate("1970-01-01T00:00:00Z"),
                        "optimeDurableDate" : ISODate("1970-01-01T00:00:00Z"),
                        "lastAppliedWallTime" : ISODate("1970-01-01T00:00:00Z"),
                        "lastDurableWallTime" : ISODate("1970-01-01T00:00:00Z"),
                        "lastHeartbeat" : ISODate("2022-12-26T03:41:17.427Z"),
                        "lastHeartbeatRecv" : ISODate("1970-01-01T00:00:00Z"),
                        "pingMs" : NumberLong(0),
                        "lastHeartbeatMessage" : "",
                        "authenticated" : false,
                        "syncSourceHost" : "",
                        "syncSourceId" : -1,
                        "infoMessage" : "",
                        "configVersion" : -1,
                        "configTerm" : -1
                }
        ],

And I ensure that I follow some rules:

  • Same MongoDB version (5.0.6)
  • Same mongod.conf file (same replSet name)
  • Same keyFile and chmod 400 (read-only)
  • Can ping each order (open port ICMP inbound)

Many thanks !!

Khiem Nguyen
  • 67
  • 2
  • Use sha1sum or something similar to compare the key files, there may be a non-printing difference. – Joe Dec 26 '22 at 05:44
  • I used winscp to copy all config folder from VPS for Primary to VPS for Secondary. Then, I checked by using diffchecker tool online and result is the same! – Khiem Nguyen Dec 26 '22 at 06:25
  • New member is in internet but your existing members have a private IP address. You cannot reach a private IP from the internet, the packets are not routed by any router. – Wernfried Domscheit Dec 26 '22 at 06:41
  • Yah my PRIMARY instance is in a subnet 10.5.0.0/24 with static IP 10.5.0.11. And the SECONDARY_01 in same that subnet with IP 10.5.0.12. Now, I add SECONDARY_02 in container docker at another VPS and I expose port default 27017:27019. I run rs.add("VPS2:27019") to add secondary_02. More than I can ping VPS2_IP from exec bash in container is running PRIMARY – Khiem Nguyen Dec 26 '22 at 06:45
  • ping works different. Better try `curl --connect-timeout 10 --silent --show-error VPS2:27019`, see https://stackoverflow.com/questions/37839365/simple-http-tcp-health-check-for-mongodb/63790795#63790795 – Wernfried Domscheit Dec 26 '22 at 16:23

0 Answers0