Loop through all Mongo collections and execute query

Question

First of, I'm quite new to mongodb. Here's my question I've not been able to find a solution to.

Let's say I have 3 different collections.

mongos> show collections
collectionA
collectionB
collectionC

I want to create a script that iterates over all collections ind this database and find the last inserted timestamp in each of these collections. Here's what works inside mongos.

var last_element = db.collectionA.find().sort({_id:-1}).limit(1);
printjson(last_element.next()._id.getTimestamp());
ISODate("2014-08-28T06:45:47Z")

1. Problem (Iterate over all collections)

Is there any possibility to to sth. like.

var my_collections = show collections;
my_collections.forEach(function(current_collection){
    print(current_collection);
});

Problem here, the assignment for my_collections does not work. I get SyntaxError: Unexpected identifier. Do I need to quote the 'show' statement ? Is it even possible ?

2. Problem (storing collection in js var)

I can workaround Problem 1 by doing this:

var my_collections = ["collectionA", "collectionB", "collectionC"];
my_collections.forEach(function(current_collection){
    var last_element = db.current_collection.find().sort({_id:-1}).limit(1);
    print(current_collection);
    printjson(last_element.next()._id.getTimestamp());
});

The last_element.next() produces the following error:

error hasNext: false at src/mongo/shell/query.js:124

It seems that last_element isn't saved correctly.

Any suggestions on what I'm doing wrong??

UPDATE

Neils answer lead me to this solution. In addition to his code I had to check if the function getTimestamp really exist. For some 'virtual' collections there seem to be no _id property.

db.getCollectionNames().forEach(function(collname) {
    var last_element = db[collname].find().sort({_id:-1}).limit(1);
    if(last_element.hasNext()){
        var next = last_element.next();
        if(next._id !== undefined && typeof next._id.getTimestamp == 'function'){
           printjson(collname + " >> "+next._id.getTimestamp());
        }else{
          print(collname + " undefined!! (getTimestamp N/A)")
        }
    }
});

Just to clarify here. The only reason `.getTimeStamp()` would fail here is that you do not actually have an `ObjectId` in the primary key `_id` field in every document. You **should** really look at this, as though MongoDB will "allow" you to mix types in fields since it is "schemaless" you **should not** be doing this as a general practice and should really be correcting this where this occurs to make the entries consistent. — Neil Lunn, Sep 01 '14 at 11:11
Thanks for that notice! As stated I'm quite new to mongo and come from a relational db background. I'm still not sure how these collections were create nor why they have no ObjectID. As you stated I'll need to correct this and need to understand the why and how. Thanks! — cb0, Sep 01 '14 at 20:28

score 59 · Accepted Answer · edited Jun 16 '15 at 09:50

59

There is the db.getCollectionNames() helper method that does this for you. You can then implement your code:

db.getCollectionNames().forEach(function(collname) {
    // find the last item in a collection
    var last_element = db[collname].find().sort({_id:-1}).limit(1);
    // check that it's not empty
    if (last_element.hasNext()) {
        // print its timestamp
        printjson(last_element.next()._id.getTimestamp());
    }
})

You probably also want a .hasNext() check in there to cater for possible empty collections.

edited Jun 16 '15 at 09:50

Ron Wertlen

830
10
24

answered Aug 28 '14 at 07:55

Neil Lunn

148,042
36
346
317

I'd say this is really only safe to do in the shell environment where all calls are synchronous; this solution in an asynchronous driver environment is risky (many parallel queries in rapid succession). – zamnuts Sep 01 '14 at 07:46
@zamnuts There are plenty of ways to safely process lists and data streams in an async environment. You just need to be aware of callbacks to announce each iteration. If you're not sure about this then you can always ask a question, and someone will likely show you how to do it. – Neil Lunn Sep 01 '14 at 08:15

score 0 · Answer 2 · answered May 03 '22 at 13:00

Rename the collection name present in all the records using the following script:

    db = db.getSiblingDB("admin");
dbs = db.runCommand({ "listDatabases": 1 }).databases;
dbs.forEach(function(database) {
    db = db.getSiblingDB(database.name);
    db.currentname.renameCollection("newname");
});

Loop through all Mongo collections and execute query

2 Answers2

Linked

Related