How to return value of db calls from a asynchronous function

Question

I was looking for an answer and found this. Like in jquery there is when function. Using some guidelines to synchronize my asynchronous calls I am using async.js in Node Here is the code.

async.forEach(dateInDateFormat, function(item, callback) {
        console.log(moment(item.startDate).toDate());
        var result = Trip.aggregate([{
            "$unwind": "$trips"
        }, {
            "$match": {
                "trips.startTime": {
                    "$gte": moment(item.startDate).toDate(),
                    "$lte": moment(item.endDate).toDate()
                }
            }
        }, {
            "$group": {
                "_id": {
                    "date": {
                        "$dayOfMonth": "$trips.startTime"
                    }
                },
                "distance": {
                    "$sum": "$trips.distance"
                }
            }
        }]);
        result.exec(function(err, doc) {
            console.log(doc);
        });
        console.log("Executing callback");
        callback();
    }, function(error) {
        console.log("Loop over");
    });

This is the output.

Executing callback
Loop over
[ { _id: { date: 23 }, distance: 0 },
{ _id: { date: 22 }, distance: 0 },
{ _id: { date: 21 }, distance: 0 } ]
[ { _id: { date: 29 }, distance: 210 },
  { _id: { date: 27 }, distance: 210 },
  { _id: { date: 26 }, distance: 210 },
  { _id: { date: 25 }, distance: 0 } ]
[]

As you can see. Executing callback and Loop over are executed before I get the data. I cannot send it back to my Angular Frontend using res.send(). One answer is you cannot return a value from a asynchronous function. You only return a callback. How do I do that in this case? Or How do I use the async.series([]) function. Because I used that also and even that is not giving me exact results. Thanks

EDIT Answer in short, the position of callback function matters. If you give it at wrong place don't expect write answers. Please see the code in the answer and comment thread for better explanation.

dm03514 · Accepted Answer · 2015-11-02T23:58:51.650

The callback is kind of magic and just indicates to async.foreach that a specific foreach execution has completed. I am not familiar with the aggregate function or the exec, so I'm assuming that aggregate builds a query, and exec is executing that query asynchronously.

The forEach callback could be moved to the result.exec result. This would indicate to async that the execution of that particular query is over. When all executions are over the function passed as the second param to forEach will be executed. When it is executed, all functions have completed or an error has occurred, in which case you can send to res.

The overall flow:

Create a list of data to operate on asyncronously
Execute a function (2nd param to async.forEach) against each item in your list asyncronously.
1. build aggregation query
2. exeucte aggregation query asynchronously
3. Indicated to async.foreach that aggregate operation has been completed by calling callback function
When callback has been called for each item in the array async will call the function that was passed as 3rd param to async.forEach.
Return result to client because all tasks have been resolved.

async.forEach(dateInDateFormat, function(item, callback) {
        console.log(moment(item.startDate).toDate());
        var result = Trip.aggregate([{
            "$unwind": "$trips"
        }, {
            "$match": {
                "trips.startTime": {
                    "$gte": moment(item.startDate).toDate(),
                    "$lte": moment(item.endDate).toDate()
                }
            }
        }, {
            "$group": {
                "_id": {
                    "date": {
                        "$dayOfMonth": "$trips.startTime"
                    }
                },
                "distance": {
                    "$sum": "$trips.distance"
                }
            }
        }]);
        result.exec(function(err, doc) {
            console.log(doc);
            console.log("Executing callback");
            callback();
        });
    }, function(error) {
        // all tasks passed to forEach have been completed, or 
        // an error has occurred
        res.send('COMPLETE');
        console.log("Loop over");
    });

It's crazy to me how a simple operation can have such a crazy programming flow :)

Well thanks for the explanation, but it's still not clear. If you see the output (I just edited the output) and see the console.logs() in question. The console.log("Loop over")is executed well before the array is printed. So the idea of sending `res.send` at `function(error)` won't work. Do you know any other work around? — Saras Arya, Nov 01 '15 at 18:25
@SarasArya did you try my answer? In your example, i'm pretty sure, the "Loop over" is executed so quickly because you aren't waiting on the response of `result.exec` — dm03514, Nov 01 '15 at 21:19
Hey It worked... Looking at the code first. I thought it's the same. After your second comment. I ran it through a diffChecker and found out that the position of callback is changed which actually makes sense. It was difficult to understand the answer at first, but the working of code explained me how. Thanks a lot. That will go down as one of the most important things in my history of asynchronous programming. — Saras Arya, Nov 02 '15 at 22:58

How to return value of db calls from a asynchronous function

1 Answers1