Chaining an arbitrary number of promises in Q

Question

I want to send an HTTP request N times. I want to eventually have information about the results of each of those requests.

Running the request function once works great. Here's the HTTP request function using Q.defer():

function runRequest() {
    var deferred = Q.defer(),
        start = (new Date).getTime(),
        req = HTTP.request(options, function(res) {
            var end = (new Date).getTime(),
            requestDetails = {
                reqStatus: res.statusCode,
                reqStart: start,
                reqEnd: end,
                duration: end - start
            }
            deferred.resolve(requestDetails);
        });
    req.on('error', function(e) {
        deferred.reject(e.message);
    });
    req.end();

    return deferred.promise;
}

If I do this, I get back the data I expect:

runRequest().then(function(requestDetails) {
    console.log('STATUS: ' + requestDetails.reqStatus);
    console.log('Duration: ' + requestDetails.duration);
    console.log('Start: ' + requestDetails.reqStart);
    console.log('End: ' + requestDetails.reqEnd);

}, function(error) {
    console.log('Problem with request: ' + error);
})
.done();

To iterate, I tried to fit that into a for loop:

function iterateRequests() {
    var deferred = Q.defer();
    var reqResults = [];
    for (var iteration = 0; iteration < requests; iteration++) {
        runRequest()
        .then(function(requestDetails) {
            console.log('STATUS: ' + requestDetails.reqStatus);
            reqResults.push(requestDetails);
        }, function(error) {
            console.log('Problem with request: ' + error);
        });
    }
    deferred.resolve(reqResults);
    return deferred.promise;
}

Then I call it like this:

iterateRequests()
.then(function(results) {
    console.log(results);
    console.log("in the success callback after iterateRequests");
}, function() {
    console.log("in the failure callback after iterateRequests");
})
.done();

I end up getting into the success callback (i.e., it logs "in the success callback after iterateRequests"). However, the console.log(results) prints before I get the logs from runRequest().then() callback and it's an empty array.

Any ideas or some guidance on chaining/iterating over promise-return functions?

Thanks!

Update Follow up question in response to @abject_error's answer:

Checked out Q.all. Definitely looks like what I need. And it's much simpler that what I was working with. I made a simple test case to help me figure out how it works:

var Q = require("q");

function returner(number) {
    var deferred = Q.defer();

    deferred.resolve(number);
    return deferred.promise;
}

function parent() {
    return Q.all([
        returner(1),
        returner(2),
        returner(4)
    ]);
}


parent()
.then(function(promises) {
    // works - promises gives me [1, 2, 4]
    console.log(promises);
});

So I see how I can use it if I know beforehand the number of times I need to call it (and which functions I'm going to call). Any tips on how to get a dynamic number of calls to returner (in this example) or runRequest (in my original example) in the array?

score 7 · Accepted Answer · answered Jul 16 '13 at 23:11

7

This answers the update part of the question:

var buildCalls = function() {

  var calls = [];
  for (var i in stories) {

    calls.push(myFunc(i));
  }
  return calls;
}

return Q.all(buildCalls());

answered Jul 16 '13 at 23:11

mkulke

496
3
5

Took me a while to get to this. Thanks for the answer. It was exactly what I needed. – salsbury Oct 17 '13 at 03:29

score 5 · Answer 2 · edited Oct 07 '13 at 07:31

5

Q has other functions to aid in Promise based workflows. The method you need to use is Q#all. If you have an array of promises, and you want to call a function when all of them have successfully fulfilled, you do

Q.all(array_of_promises).then(success_callback, failure_callback);

After all the request promises are fulfilled, success_callback is called. If any of them rejects, the failure_callback is called immediately.

edited Oct 07 '13 at 07:31

Domenic

110,262
41
219
271

answered Jun 07 '13 at 05:10

abject_error

2,858
1
19
23

Thanks abject_error. Looks like exactly what I need. Any suggestions on how to get a dynamic number of function calls in array_of_promises? I added some further explanation to my question above. – salsbury Jun 07 '13 at 07:03
@Domenic - hi, is there any (reasonable) limit on the size of the array - will Q internally throttle it so it doesn't overstep the bounds of how many asynchronous tasks can be running at the same time on machine. For instance, is 10K items going to be batched / throttled ? – arcseldon Nov 20 '14 at 15:31
Most asynchronous operations are naturally batched or throttled by the underlying system, e.g. most network will pipeline open connections after a certain number. – Domenic Nov 20 '14 at 18:53

Chaining an arbitrary number of promises in Q

2 Answers2

Linked