13

Three members, primary & secondary - and a third one that's "OTHER" - I can't find any info on that state, not sure what to do, I've restarted the instance, but it always comes-up the same. Can find no documentation on that state.

I'm new to replica sets - and any help would be appreciated.

Jolly
  • 349
  • 1
  • 2
  • 9

5 Answers5

30

The config is not set correctly.

You can use following command to init:

rs.initiate({
      _id: "rs0",
      version: 1,
      members: [
         { _id: 0, host : "localhost:27017" }
      ]
   }
)

If you have already initiated, you may get the error msg like me:

singleNodeRepl:OTHER> rs.initiate({ _id: "rs0", members: [ { _id: 0, host : "localhost:27017" } ] } )
{
    "info" : "try querying local.system.replset to see current configuration",
    "ok" : 0,
    "errmsg" : "already initialized",
    "code" : 23,
    "codeName" : "AlreadyInitialized"
}

The solution is to reconf the mongo:

singleNodeRepl:OTHER> rsconf = rs.conf()
singleNodeRepl:OTHER> rsconf.members = [{_id: 0, host: "localhost:27017"}]
[ { "_id" : 0, "host" : "localhost:27017" } ]
singleNodeRepl:OTHER> rs.reconfig(rsconf, {force: true})
{ "ok" : 1 }
singleNodeRepl:OTHER>
singleNodeRepl:SECONDARY>
singleNodeRepl:PRIMARY>
Yihe
  • 4,094
  • 2
  • 19
  • 21
7

I was able to replicate the scenario and this is what I got when I did rs.status() from the mongod where the command prompt was showing OTHER:

{
    "state" : 10,
    "stateStr" : "REMOVED",
    "uptime" : 41,
    "optime" : {
        "ts" : Timestamp(1518192445, 1),
        "t" : NumberLong(26)
    },
    "optimeDate" : ISODate("2018-02-09T16:07:25Z"),
    "ok" : 0,
    "errmsg" : "Our replica set config is invalid or we are not a member of it",
    "code" : 93,
    "codeName" : "InvalidReplicaSetConfig",
    "operationTime" : Timestamp(1518192445, 1),
    "$clusterTime" : {
        "clusterTime" : Timestamp(1518193246, 1),
        "signature" : {
            "hash" : BinData(0,"AAAAAAAAAAAAAAAAAAAAAAAAAAA="),
            "keyId" : NumberLong(0)
        }
    }
}

Let's assume the faulty mongod is the 3rd member (index 2) in the members array of the JSON you get when you do rs.conf(). Go to your primary and remove the faulty mongod from the replica set:

rsconf = rs.conf()
rsconf.members = [rsconf.members[0], rsconf.members[1]] //index 0 and 1 are your working members, we're omitting the 3rd member which is faulty.
rs.reconfig(rsconf);

Now restart your mongod that was showing as OTHER. And go to your primary again and add the member again. Let's assume the IP is 10.00.00.00 and port is 27019:

rs.add("10.00.00.00:27019")

This fixed the issue, and the status changed from OTHER to SECONDARY.

Note that reconf will reset all the client connections. You may need a maintenance window of about a minute to do the reconfigurations.

Andrew Nessin
  • 1,206
  • 2
  • 15
  • 22
  • At the end, I also needed to do the same `rs.add` command on the member being added since it picked up the config without it as a member after the restart. – Muhd Apr 19 '19 at 05:24
3

For posterity - the problem is that the instance can't sync with the primary or secondary because it was down too long. It goes into a state of trying primary and secondary, over-and-over, never being allowed to sync. This eventually created an 8GB log file, which I couldn't open, so I couldn't see the problem at the time. Solution apparently is to stop the errant mongo instance - dump its data - and start it again, as if it was a new member of the replica set.

Jolly
  • 349
  • 1
  • 2
  • 9
0

change the priority

Let's assume you have the three members

rs0:OTHER> cfg = rs.conf()

rs0:OTHER> cfg.members[0].priority = 1

rs0:OTHER> cfg.members[1].priority = 0.5

rs0:OTHER> cfg.members[2].priority = 0.5

rs0:OTHER> rs.reconfig(cfg, {force: true})

rs0:OTHER> rs.config()

rs0:PRIMARY>
Book Of Zeus
  • 49,509
  • 18
  • 174
  • 171
San n
  • 11
  • 2
  • 1
    Providing some explanation around your answer, will make it more helpful. Also suggest providing what your thinking is. – Ajay M Jun 10 '19 at 04:20
0

The solution for me was : disconnection from robot3t and all mongodb Gui (compass..) used on my pc. then when i reopen the mongo shell it connects as primary.