Mongoose: how to get data from a capped collection in an express.js app?

Question

I'd like to listen on a MongoDB capped collection, using it as a logging facility.
I use node, express.js, mongo (with mongoose).

This is the (simplified) code I come with up to now:

var mongoose = require('mongoose');

mongoose.connect('mongodb://localhost/mydb');

var logSchema = new mongoose.Schema({
  date: Date,
  message: String
}, {
  capped: {
    size: 1024
  }
});
var Log = mongoose.model('Log', logSchema);

var filter = { "date": { "$gte": Date.now() } };
var stream = Log.find(filter).tailable().stream();
stream.on('data', function(doc) {
  console.log('log stream data - new doc:', doc.message);
}).on('error', function (error) {
  console.log('status stream data - error:', error.message);
}).on('close', function () {
  console.log('status stream data - closed');
});

// ...

var log = new Log();
logger = function(message) {
  log.date = new Date();
  log.message = message;
  log.save(function(err) {
    if (err) {
      return console.error('error saving log');
    }
    console.log('log message "' + message + '" added');
  });
};

// ...

myRoutingMethod = function(req, res) {
  logger('my routing method started');
  // ...
  res.json('done');
});

My problem is, before myRoutingMethod() is called, I get:

database connection opened
log message "my new message" added
status stream data - error: No more documents in tailed cursor
status stream data - closed

So, I never get

log stream data - new doc: my new message

I am probably missing something about integration of stream() on capped Log collection with express.js...

Any clue?

P.M · Accepted Answer · 2019-03-26T19:58:08.447

It is hard to spot whatever went wrong code. However, based on other answers here on StackOverflow, the following may help you out:

First, check the version of Mongoose on your environment and make sure it is 2.7 or latest version.
If you had that collection in non-capped mode and added capped mode after a couple of iterations, Try to drop the collection and retry from scratch. You may need to backup the collection and re-initialize from the backup.
based on initializations found in docs here on StackOverflow, I would suggest configuring the capped collection schema as following:

//new Schema declaration 
var logSchema = mongoose.Schema({...},{capped:{size: 1024, max: 1000,autoIndexId: true}});

//Export your model as following
 module.exports = mongoose.model('Log', logSchema);

To initialize and use your Mongoose:

var  Log = require(path/to/log/schema);
var query = { /** query paremeters here*/ };

//Initialize the stream
var stream = Log.find(query).tailable().stream();

//Process data
stream.on('data', function(doc){});
stream.on('error', function(error){});
stream.on('close', function(status){});

To save(or edit) operation, you may refer to the same old approach as

new Log(params).save(function(error, log){
  //Do something with the error or new log
});

You may find more information on this StackOverflow answer as well: https://stackoverflow.com/a/18045399/132610

I hope this helps.

Mongoose: how to get data from a capped collection in an express.js app?

1 Answers1