How to call function after completion of async functions inside loop?

Question

I have a forEach loop in NodeJS, iterating over a series of keys, the values of which are then retrieved asynchronously from Redis. Once the loop and retrieval has complete, I want to return that data set as a response.

My problem at the moment is because the data retrieval is asyncrhonous, my array isn't populated when the response is sent.

How can I use promises or callbacks with my forEach loop to make sure the response is sent WITH the data?

exports.awesomeThings = function(req, res) {
    var things = [];
    client.lrange("awesomeThings", 0, -1, function(err, awesomeThings) {
        awesomeThings.forEach(function(awesomeThing) {
            client.hgetall("awesomething:"+awesomeThing, function(err, thing) {
                things.push(thing);
            })
        })
        console.log(things);
        return res.send(JSON.stringify(things));
    })

There are many QA on this problem. Some are listed on the right of this page. — Denys Séguret, Apr 01 '14 at 14:46
Right now, I'd suggest to use a promise library. [Here's an introduction](http://margaine.com/bluebird/) — Denys Séguret, Apr 01 '14 at 14:47
@dystroy you should consider adding a promise answer to that question. The current one is rather ugly. Also, something something asynchronous semaphore. — Benjamin Gruenbaum, Apr 01 '14 at 14:53

score 11 · Accepted Answer · edited May 23 '17 at 11:51

I use Bluebird promises here. Note how the intent of the code is rather clear and there is no nesting.

First, let's promisify the hgetall call and the client -

var client = Promise.promisifyAll(client);

Now, let's write the code with promises, .then instead of a node callback and aggregation with .map. What .then does is signal an async operation is complete. .map takes an array of things and maps them all to an async operation just like your hgetall call.

Note how Bluebird adds (by default) an Async suffix to promisifed methods.

exports.awesomeThings = function(req, res) {
    // make initial request, map the array - each element to a result
    return client.lrangeAsync("awesomeThings", 0, -1).map(function(awesomeThing) {
       return client.hgetallAsync("awesomething:" + awesomeThing);
    }).then(function(things){ // all results ready 
         console.log(things); // log them
         res.send(JSON.stringify(things)); // send them
         return things; // so you can use from outside
    });
};

Thanks, this is perfect. I knew promises would be the way to go for this, but a lot of the solutions I've found (since posting this question) have been very... verbose. — Anonymous, Apr 01 '14 at 14:59
Glad I could help. Promises are a really powerful abstraction and really simplify code nicely. They also have built in debuggability - if any of the code above has an error - you'll know with Bluebird promises. They chain well, they compose nicely and contain the concurrency primitives you actually need. They are also a part of JavaScript as of EcmaScript 6 - the next standard so it's fair to assume code using them is very future compatible. — Benjamin Gruenbaum, Apr 01 '14 at 15:09

score 1 · Answer 2 · answered Apr 02 '14 at 14:24

No lib is needed. Easy as pie, it's just an async loop. Error handling is omitted. If you need to do a parallel async loop use a counter.

exports.awesomeThings = function(req, res) {
    client.lrange("awesomeThings", 0, -1, function(err, awesomeThings) {
        var len = awesomeThings.length;
        var things = [];
        (function again (i){
            if (i === len){
                //End
                res.send(JSON.stringify(things));
            }else{
                client.hgetall("awesomething:"+awesomeThings[i], function(err, thing) {
                    things.push(thing);

                    //Next item
                    again (i + 1);
                })
            }
        })(0);
});

How to call function after completion of async functions inside loop?

2 Answers2

Linked

Related