MongoDB - find() calls getting stuck sometimes - Timeout

Question

We are using MongoDb for saving and fetching data.

All calls that are putting data into collections are working fine and are through common method.

All calls that are fetching data from collections are working fine sometimes and are through common method.

But Sometimes, only for one of the collection, i get my calls being stuck for forever, consuming CPU usage. I have to manually kill the threads otherwise it consumes my whole CPU.

Mongo Connection

MongoClient mongo = new MongoClient(hostName , Integer.valueOf(port));
DB mongoDb = mongo.getDB(dbName);

Code To fetch

DBCollection collection = mongoDb.getCollection(collectionName);
DBObject dbObject = new BasicDBObject("_id" , key);
DBCursor cursor = collection.find(dbObject);

Though i have figured out the collection for which it is causing issues, but how can i improve upon this, since it is occurring for this particular collection and sometimes.

EDIT

Code to save

DBCollection collection = mongoDb.getCollection(collectionName);
DBObject query = new BasicDBObject("_id" , key);
DBObject update = new BasicDBObject();
update.put("$set" , JSON.parse(value));
collection.update(query , update , true , false);

Bulk Write / collection

DB mongoDb = controllerFactory.getMongoDB();
        DBCollection collection = mongoDb.getCollection(collectionName);

        BulkWriteOperation bulkWriteOperation = collection.initializeUnorderedBulkOperation();

        Map<String, Object> dataMap = (Map<String, Object>) JSON.parse(value);

        for (Entry<String, Object> entrySet : dataMap.entrySet()) {
            BulkWriteRequestBuilder bulkWriteRequestBuilder = bulkWriteOperation.find(new BasicDBObject("_id" ,
                    entrySet.getKey()));

            DBObject update = new BasicDBObject();
            update.put("$set" , entrySet.getValue());

            bulkWriteRequestBuilder.upsert().update(update);
        }

How can i set timeout for fetch calls..??

how many documents are in that particular collections? Please try running .explain() with you query which examine how query is performing, whether it is using index or not. — Atish, May 18 '16 at 06:29
Have you tried with the `MongoDatabase` and `MongoCollection` classes? It's the proposed method they proposing, instead of `DBCollection` and `DBObject`. — Lefteris008, May 18 '16 at 06:39
@Lefteris008 no i have not tried with those classes, can you please provide an example for the same. — Ankur Singhal, May 18 '16 at 06:42
@ankur-singhal: One other way is to run query at the server side using mongo shell to check if its running slow on the server itself. Further you can use .explain() to understand more about it — Atish, May 18 '16 at 06:49

score 1 · Answer 1 · edited Nov 03 '19 at 15:18

Just some possible explanations/thoughts.

In general "query by id" has to be fast since _id is supposed to be indexed, always. The code snippet looks correct, so probably the reason is in mongo itself. This leads me to a couple of suggestions:

Try to connect to mongo directly from the command line and run the "find" from there. The chances are that you'll still be able to observe occasional slowness.

In this case:

Maybe its about the disks (maybe this particular server is deployed on the slow disk or at least there is a correlation with some slowness of accessing the disk).
Maybe your have a sharded configuration and one shard is slower than others
Maybe its a network issue that occurs sporadically. If you run mongo locally/on staging env. with the same collection does this reproduce?
Maybe (Although I hardly believe that) the query runs in sub un-optimal way. In this case you can use "explain()" as someone has already suggested here.
If you happen to have replica set, please figure out what is the [Read Preference]. Who knows, maybe you prefer to get this id from the sub-optimal server

it is simply deployed on single node, no sharding, no replicas, just for one of the collection i am observing this issue, and that that too not always — Ankur Singhal, May 18 '16 at 07:05

Lefteris008 · Answer 2 · 2016-05-18T07:21:09.727

A different approach is to use the proposed method for MongoDB 3.2 Driver. Keep in mind that you have to update your .jar libraries (if you haven't) to the latest version.

public final MongoClient connectToClient(String hostName, String port) {
    try {
        MongoClient client = new MongoClient(hostName, Integer.valueOf(port));
        return client;
    } catch(MongoClientException e) {
        System.err.println("Cannot connect to Client.");
        return null;
    }
}

public final MongoDatabase connectToDB(String databaseName) {
    try {
        MongoDatabase db = client.getDatabase(databaseName);
        return db;
    } catch(Exception e) {
        System.err.println("Error in connecting to database " + databaseName);
        return null;
    }

public final void closeConnection(MongoClient client) {
    client.close();
}

public final void findDoc(MongoDatabase db, String collectionName) {
    MongoCollection<Document> collection = db.getCollection(collectionName);
    try {
        FindIterable<Document> iterable = collection
                .find(new Document("_id", key));
        Document doc = iterable.first();
        //For an Int64 field named 'special_id'
        long specialId = doc.getLong("special_id");
    } catch(MongoException e) {
        System.err.println("Error in retrieving document.");
    } catch(NullPointerException e) {
        System.err.println("Document with _id " + key + " does not exist.");
    }
}

public final void insertToDB(MongoDatabase db, String collectioName) {
    try {
        db.getCollection(collectionName).insertOne(new Document()
            .append("special_id", 5)
            //Append anything
        );
    catch(MongoException e) {
        System.err.println("Error in inserting new document.");
    }
}

public final void updateDoc(MongoDatabase db, String collectionName, long id) {
    MongoCollection<Document> collection = db.getCollection(collectionName);
    try {
        collection.updateOne(new Document("_id", id),
                             new Document("$set", 
                                     new Document("special_id", 
                                             7)));
    catch(MongoException e) {
        System.err.println("Error in updating new document.");
    }
}

public static void main(String[] args) {
    String hostName = "myHost";
    String port = "myPort";
    String databaseName = "myDB";
    String collectionName = "myCollection";
    MongoClient client = connectToClient(hostName, port);
    if(client != null) {
        MongoDatabase db = connectToDB(databaseName);
        if(db != null) {
            findDoc(db, collectionName);
        }
        client.closeConnection();
    }
}

EDIT: As the others suggested, check from the command line if the procedure of finding the document by its ID is slow too. Then maybe this is a problem with your hard drive. The _id is supposed to be indexed but for better or for worse, re-create the index on the _id field.

i am working on your solution, need to change code for write and bulk write calls. I have edited my question — Ankur Singhal, May 18 '16 at 07:02
@ankur-singhal I have mistakenly added a `finally` block the closes the client when connecting to database, please remove it and place it inside the `catch` block. — Lefteris008, May 18 '16 at 07:05
i need to change for write and bulk write operation, please help. — Ankur Singhal, May 18 '16 at 07:09
@ankur-singhal I have completely changed my code, separating everything into new methods and creating a new `insertToDB` as you asked. — Lefteris008, May 18 '16 at 07:17
Thanks, i am following up on your answer, would you mind helping me out for bulkupdate — Ankur Singhal, May 18 '16 at 07:26
Let us [continue this discussion in chat](http://chat.stackoverflow.com/rooms/112234/discussion-between-ankur-singhal-and-lefteris008). — Ankur Singhal, May 18 '16 at 07:28
@ankur-singhal I would love to, but I have work to do and I won't be available for the next few hours. Please, check the answer to [this question](http://stackoverflow.com/questions/35846474/how-to-perform-a-bulk-update-of-documents-in-mongodb-with-java); you have to update everything to comply with the Jave 3.2 Driver. Also, keep in mind that this is not a solution, we are trying to identify if the problem continues while using the new driver. You may change everything and the problem might still be here. — Lefteris008, May 18 '16 at 07:30
i did complete coding, just to find we are running on 2.2 mongodb , and we cannot update it as it is supporting other applications as well. Is there any option to set timeout while reading from mongo. — Ankur Singhal, May 18 '16 at 08:09
@ankur-singhal As this is a fast-paced release, MongoDB 2.2 is an old version; maybe the problem you are referring to, is a bug that got fixed in later versions. But as your code relies on this version of the library, there's not much you can do, except for the timeouts. — Lefteris008, May 18 '16 at 08:17
exactly, i am not able to set the timeout, how can i do that in my above code — Ankur Singhal, May 18 '16 at 08:35

score 1 · Accepted Answer · answered May 20 '16 at 10:16

1

The answers posted by others are great, but did not solve my purpose. Actually issue was in my existing code itself , my cursor was waiting in while loop infinite time.

I was missing few checks which has been resolved now.

answered May 20 '16 at 10:16

Ankur Singhal

26,012
16
82
116

Can you explain what checks? – Prateek May 05 '22 at 08:58

MongoDB - find() calls getting stuck sometimes - Timeout

3 Answers3