Mongodb: how to create a `tail -f` view on a capped collection?

Question

I would like to create a sort of a 'dashboard' on a capped collection (which is used as a log table) on my Mongo database. This is how I create the collection:

db.createCollection( "messages", { capped: true, size: 100000 } );

I do a collection.find(), with options tailable:true, awaitdata:true, and numberOfRetries:-1 (infinite retries).

What puzzles me is that I'd expect the find().each() loop to wait for new data (messages)... instead (after a few seconds) it errors out (with No more documents in tailed cursor... :-()

This is the code I'm working with:

var mongo = require('mongodb');  
mongo.MongoClient.connect('mongodb://127.0.0.1/myDb', function (err, db) {
  db.collection('messages', function(err, collection) {
    if (err) {
      return console.error('error in status collection:', err);
    }
    collection.find( // open tailable cursor
      {},
      { tailable: true, awaitdata: true, numberOfRetries: -1 }
    ).each(function(err, doc) {
      if (err) {
        if (err.message === 'No more documents in tailed cursor') {
          console.log('finished!');
        } else {
          console.error('error in messages collection:', err);
        }
      } else {
        if (doc) {
          console.log('message:', doc.message);
        }
      }
    })
  });
});

What do I miss?

UPDATE:

Not having received any conclusive answer until now, I deduce MongoDb tailable collections is not ready for prime time... :-(((

Sadly giving up for a more classic and robust fs logging solution...

score 2 · Accepted Answer · edited Oct 12 '15 at 13:45

2

You could set up subscriber function that subscribes for new MongoDB documents using the tailable find() cursor as a node.js stream. The following demonstrates this:

// subscriber function
var subscribe = function(){

    var args = [].slice.call(arguments);
    var next = args.pop();
    var filter = args.shift() || {};

    if('function' !== typeof next) throw('Callback function not defined');

    var mongo = require('mongodb');  
    mongo.MongoClient.connect('mongodb://127.0.0.1/myDb', function(err, db){

        db.collection('messages', function(err, collection) {           
            var seekCursor = collection.find(filter).sort({$natural: -1}).limit(1);
            seekCursor.nextObject(function(err, latest) {
                if (latest) {
                    filter._id = { $gt: latest._id }
                }           

                var cursorOptions = {
                    tailable: true,
                    awaitdata: true,
                    numberOfRetries: -1
                };

                var stream = collection.find(filter, cursorOptions).sort({$natural: -1}).stream();
                stream.on('data', next);
            });
        });
    });

};

// subscribe to new messages
subscribe( function(document) {
    console.log(document);  
});

Source: How to subscribe for new MongoDB documents in Node.js using tailable cursor

edited Oct 12 '15 at 13:45

MarcoS

17,323
24
96
174

answered Oct 12 '15 at 13:23

chridam

100,957
23
236
235

1

Thanks (my bad!, I did already find that solution on the web, but forgot about it... :-(). Do you think it could be possible to implement the same logic with `mongoose`, instead of `mongodb`? – MarcoS Oct 12 '15 at 13:40
1

Yes, quite possible. Give this a go https://github.com/Automattic/mongoose/issues/3109 – chridam Oct 12 '15 at 13:43
Thanks. The answer is correct, but it does not fully answers the question: How to create .. a **view**... For example, in an Angular.js view, how should I call subscribe() ? – MarcoS Oct 12 '15 at 13:50
I think something like that has been done in this Blog post [**How to create a pub/sub application with MongoDB ? Introduction**](http://tugdualgrall.blogspot.co.uk/2015/01/how-to-create-pubsub-application-with.html) but using sockets. – chridam Oct 12 '15 at 13:54
I did see that post already, too... :-) The problem is that it works with sockets, which I have to avoid.... Do you think a different solution is possible? – MarcoS Oct 12 '15 at 13:56
1

Hehehehehe, we keep bumping heads in the same posts :P Let me see if I can come up with something in the long-run. – chridam Oct 12 '15 at 13:58
Let us [continue this discussion in chat](http://chat.stackoverflow.com/rooms/92062/discussion-between-marcos-and-chridam). – MarcoS Oct 12 '15 at 14:30

score 0 · Answer 2 · answered May 20 '21 at 18:27

Maybe someone also looking for a bare-mongo-terminal solution (I mean, without live listening on any programming language). If you want just once look at the end of collection, consider use this

db.oplog.rs.find().sort({$natural: -1})

Hope I have helped someone :)

Mongodb: how to create a `tail -f` view on a capped collection?

2 Answers2