Getting data out of a MongoDB call

Question

I am unable to retrieve data from my calls to MongoDB. The calls work as I can display results to the console but when I try to write/copy those results to an external array so they are usable to my program, outside the call, I get nothing. EVERY example that I have seen does all it's work within the connection loop. I cannot find any examples where results are copied to an array (either global or passed in), the connection ends, and the program continues processing the external array.

Most of the sample code out there is either too simplistic (ie. just console.log within the connection loop) or is way too complex, with examples of how to make express api routes. I don't need this as I am doing old fashioned serial batch processing. I understand that Mongo is built to be asynchronous but I should still be able to work with it.

MongoClient.connect('mongodb://localhost:27017/Lessons', function (err, db) {
    assert.equal(err, null);
    console.log("Connectied to the 'Lessons' database");
    var collection = db.collection('students');
    collection.find().limit(10).toArray(function(err, docs) {
        // console.log(docs);
        array = docs.slice(0);    //Cloning array
        console.log(array);
        db.close();
    });
 });
console.log('database is closed');
console.log(array);

It looks like I'm trying to log the data before the loop has finished. But how to synchronize the timing? If somebody could explain this to me I'd be really grateful as I've been staring at this stuff for days and am really feeling stupid.

Answer 1

From the code you have shared, do you want the array to display in the console.log at the end? This will not work with your current setup as the 2 console.log's at the end will run before the query to your database is complete.

You should grab your results with a call back function. If your not sure what those are, you will need to learn as mongo / node use them everywhere. Basically javascript is designed to run really fast, and won't wait for something to finish before going to the next line of code.

This tutorial helped me a lot when I was first learning: https://zellwk.com/blog/crud-express-mongodb/

Could you let us know what environment you are running this mongo request? It would give more context because right now I'm not sure how you are using mongo.

Answer 2

thanks for the quick response.

Environment is Windows7 with an instance of mongod running in the background so I'm connecting to localhost. I'm using a db that I created but you can use any collection to run the sample code.

I did say I thought it was a timing thing. Your observation "the 2 console.log's at the end will run before the query to your database is complete" really clarified the problem for me.

I replaced the code at the end, after the connect() with the following code:

function waitasecond(){
    console.log(array);
}
setTimeout(waitasecond, 2000);

And the array is fully populated. This suggests that what I was trying to do, at least the way I wanted to do, is not possible. I think I have two choices.

1. Sequential processing (as I originally concieved) - I would have to put some kind of time delay to let the db call finish before commencing.

2. Create a function with all the code for the processing that needs to be done and call it from inside the database callback when the database returns the data.

The first options is a bit smelly. I wouldn't want to see that in production so I guess I'll take the second option.

Thanks for the recommeded link. I did a quick look and the problem, for me, is that this is describing a very common pattern that relies on express listening for router calls to respond. The processing I'm doing has nothing to do with router calls.

Ah for the good old days of syncronous io.