Random CursorNotFound error with MongoDB

Question

I'm using a cursor to iterate through >30k documents in NodeJS with mongodb and write other documents during this iteration in processFn.

Example code:

const cursor = collection
    .find({}, { timeout: false })
    .addCursorFlag('noCursorTimeout', true);

while (await cursor.hasNext()) {
    const doc = await cursor.next();
    await processFn({ id: doc._id.toHexString(), ...doc });
}

await cursor.close();

For an unknown reason I get random CursorNotFound (43) errors.

{   MongoError: cursor id 54902755976 not found
name: 'MongoError',
message: 'cursor id 54902755976 not found',
ok: 0,
errmsg: 'cursor id 54902755976 not found',
code: 43,
codeName: 'CursorNotFound' }

at /###/node_modules/mongodb-core/lib/connection/pool.js:593:63
at authenticateStragglers (/###/node_modules/mongodb-core/lib/connection/pool.js:516:16)
at Connection.messageHandler (/###/node_modules/mongodb-core/lib/connection/pool.js:552:5)
at emitMessageHandler (/###/node_modules/mongodb-core/lib/connection/connection.js:309:10)
at Socket.<anonymous> (/###/node_modules/mongodb-core/lib/connection/connection.js:452:17)
at Socket.emit (events.js:180:13)
at addChunk (_stream_readable.js:274:12)
at readableAddChunk (_stream_readable.js:261:11)
at Socket.Readable.push (_stream_readable.js:218:10)
at TCP.onread (net.js:581:20)

I already checked that they are not timeouts. They always happen after less then 5 minutes. Also cursor.metrics indicates that only one cursor is open and this cursor has noCursorTimeout enabled.

I also checked that processFn resolves correctly.

I read about session and connection timeouts and also increased these to extremely high numbers for testing. Still no change.

What else could be the reason and how to debug this error?

score 2 · Accepted Answer · answered Jul 13 '18 at 19:42

2

I was researching the same error and came across this:

https://jira.mongodb.org/browse/SERVER-34810

It appears that it is a server issue that requires upgrading mongo to at least 3.6.6, 4.0.1, or 4.1.1.

Alternatively very low batch sizes (~20) seems to mitigate the issue somewhat.

answered Jul 13 '18 at 19:42

Michael Cook

1,676
2
26
47

1

Thanks for the answer! :) This might be the reason indeed. I'll check this out. – Florian Richter Jul 15 '18 at 07:03

crifan · Answer 2 · 2021-08-31T00:54:50.077

latest (2021 year) answer:

create session then associate all operations with that session
then periodically refresh session

-> official demo code

var session = db.getMongo().startSession()
var sessionId = session.getSessionId().id

var cursor = session.getDatabase("examples").getCollection("data").find().noCursorTimeout()
var refreshTimestamp = new Date() // take note of time at operation start

while (cursor.hasNext()) {

  // Check if more than 5 minutes have passed since the last refresh
  if ( (new Date()-refreshTimestamp)/1000 > 300 ) {
    print("refreshing session")
    db.adminCommand({"refreshSessions" : [sessionId]})
    refreshTimestamp = new Date()
  }

  // process cursor normally

}

official doc: cursor.noCursorTimeout() — MongoDB Manual

-> more detail can refer another post answer

score 0 · Answer 3 · answered Jan 13 '21 at 16:09

I recommend you to use the find method, you can pass an object as a second argument that receives a timeout property, it will prevent the cursor to timeout.

collection.find({}, { timeout: false })

Documentation: http://mongodb.github.io/node-mongodb-native/3.6/api/Collection.html#find

Random CursorNotFound error with MongoDB

3 Answers3