1

I'm having an issue with a script that I wrote in Node.js which interacts with a MongoDB database, using mongo.js as the connector.

My script basically does the following:

  1. read a collection from mongoDB, to get a bunch of urls

  2. go to each url, and get the data from that page

  3. perform some calculations on that data and store the calculated results in a different collection in the same DB

The issue I'm having is that after all the urls from the database are read, and all the calculations are performed, and all the calculated results are stored in a new collection, my script just hangs because the db connection has never been closed. I tried a million ways to do this, but I always close it prematurely, and it doesn't read the rest of the urls. If anyone can please help, that would be great. Thanks in advance!

Here is some code for reference (please let me know if you need more info):

db.results.drop(populateDB());

function populateDB() {
    db.urls.find({'url':{$ne:''}}).forEach(function(err, doc) {
        if (doc) {
            calculateResults(doc['url'], function(test, result) {
                db.results.save({
                    test: test,
                    result: result
                },
                function(err, saved) {
                    if (err || !saved) console.log('error on save: ' + err);
                })
            });
        } else {
            if (err) console.log(err);
        }
    });
}

function calculateResults(url, test, saveRes) {
    ...
    ...
    result = ...;
    saveRes(test, result);
    ...
}
jww
  • 97,681
  • 90
  • 411
  • 885
giantsnyc
  • 203
  • 1
  • 7
  • http://stackoverflow.com/questions/8373905/when-to-close-mongodb-database-connection-in-nodejs - Please see if this helps. – BatScream Jan 01 '15 at 23:25
  • I tried that right before posting this question :) - didn't work unfortunately. The counts won't match up because the second collection I'm inserting to has a different count. – giantsnyc Jan 01 '15 at 23:27
  • http://stackoverflow.com/questions/10551499/simplest-way-to-wait-some-asynchronous-tasks-complete-in-javascript - this one? – BatScream Jan 01 '15 at 23:29
  • That doesn't work either. I tried the async library and the Q library as well. – giantsnyc Jan 02 '15 at 17:48

0 Answers0