Is there a standard way to remove a Mongo document from a collection at a certain time using Meteor and moment.js?

Question

I'm trying to find a better way to ensure certain documents are removed from a mongo collection at a specific time, which is unique to each document. I also need to run some methods when the items are removed. I've looked into TTL indexes, but it seems they don't allow any kind of callback, and from what I read the process that removes the documents only runs once per minute, which isn't specific enough for what I need. The following is what I came up with:

var check_frequency = 30000;
Meteor.setInterval((function() {
    // figure out what elements will expire within the next check period
    var next_check = moment().add(check_frequency, 'milliseconds');
    var next_string = next_check._d.toISOString();

    var ending_items = items.find({'time_to_end': {$lt : next_string}});

    ending_items.forEach(function(db_object) {
        var time_from_now = moment(db_object.time_to_end) - moment();
        Meteor.setTimeout(function() {
            removeAndReport(db_object._id);
        }, time_from_now);

    });
}), check_frequency);

My concern is I'm not sure how Meteor.setTimeout() works with threading, so if I have hundreds or thousands of these calls I'm wondering if it will cause problems. Can anyone recommend a better way to accomplish this?

Thanks in advance.

edit: Running background jobs with Meteor or cron isn't my only concern. I realize I could accomplish the same thing with a cron job, but I'd rather not query my databases once per second to only find 3 expiring items vs. querying the database once every 30 seconds, and figure out which elements will expire in the next time period.

Answer 1

It seems like an easier solution is to store the removal date in each document, rather than the TTL. Imagine you had a collection called Messages, and each document had a removeAt field. Then you can do something like the following:

var timeout = 500;
Meteor.setInterval((function() {
  // remove any messages that should have been removed in the past
  Messages.remove({removeAt: {$lte: new Date}});
}), timeout);

Notes:

1. Make sure to index removeAt so the remove doesn't need to scan your whole collection.
2. Technically this won't breack if it runs on multiple server instances, but ideally it would only run on one. Maybe it could be run in it's own process.