Q: What is the best way in Node.js to 1) get the result of the first
async call, 2) iterate through the list, making more async calls, and
3) process the results when everything is "done".
There are multiple approaches. Hand coding, Promises, Async library. "Best" is in the eye of the beholder so not really for us to say here. I use Promises for all my async coding. They have been officially standardized in ES6 and there are good, robust implementations (I like Bluebird for the extra features it has beyond the standard that simplify complex async tasks and for it's promisifyAll() feature which gives you a promise interface on any standard async operation that uses the async callback calling convention).
I'd advise against hand coding complicated async operations because robust error handling is very difficult and exceptions can be silently eaten inside of async callbacks leading to lost error handling and difficult debugging. The Async library is probably the best non-Promise way of doing things as it provides some infrastructure and synchronization features around async callbacks.
I personally prefer promises. I think we'll see more async APIs standardizing on returning a promise as time marches forward so I think it's a better choice for a forward-looking way to learn and program.
Q: Are promises a good choice here?
Yes, promises allow you to run a bunch of asynchronous operations and then use something like Promise.all() to know when they are all done. It will also collect all the results from all the async operations for you.
Q: Is done()/next() an option?
I'm not exactly sure what you are asking about here, but you can manually code async operations to either run in parallel and know when they are done or to run them sequentially and know when they are done. Promises do a lot more of this work for you, but you can do it manually without them.
Q: Is there any "standard idiom" in Node.js for this kind of scenario?
If using promises, there would be a common way to do this. If not using promises, there is probably not a "standard idiom" as there are many different ways to code it yourself.
Promise Implementation
Here's an example using the Bluebird Promise library in node.js:
var Promise = require('bluebird');
var connection = Promise.promisifyAll(box.getConnection(req.user.login));
connection.ready(function() {
connection.getFolderItemsAsync(0, null).then(function(result) {
return Promise.map(result.entries, function(item) {
return connection.getFileInfoAsync(item.id);
})
}).then(function(results) {
// array of results here
}, function(err) {
// error here
});
});
Here's how this works:
Promisify the connection object so that all its methods have a version that returns a promise (just add "Async" onto the end of the method to call this promisified version).
Call getFolderItemsAsync() and its promise will resolve with the result.entries array
Run a map of that array, running all the operations in parallel and returning a promise that resolves with an array of ordered results when all the operations are done.
The actual result for each entry is achieved with connection.getFileInfoAsync().
Create a resolve handler and a reject handler. If any error occurs anywhere in the process, it will propagate up to the reject handler. If all operations are successful, the last resolve handler will be called with an ordered array of results.
The version above aborts if there's an error and you get no results other than the error code. If you want to continue with a null result if there's an error, then you can use something like this:
var Promise = require('bluebird');
var connection = Promise.promisifyAll(box.getConnection(req.user.login));
connection.ready(function() {
connection.getFolderItemsAsync(0, null).then(function(result) {
return Promise.map(result.entries, function(item) {
return connection.getFileInfoAsync(item.id).catch(function(err){
// post the results as null and continue with the other results
return null;
});
})
}).then(function(results) {
// array of results here (some might be null if they had an error)
}, function(err) {
// error here
});
});
Manually Coded Version
Here's a manually coded version. The key to this one is detecting when your async loop is done by comparing if (results.length === result.entries.length). Note: This has incomplete error handling which is one of the difficulties of coding this by hand and not using an async framework like promises.
var connection = box.getConnection(req.user.login);
connection.ready(function () {
connection.getFolderItems(0, null, function (err, result) {
if (err) {
// do error handling here
opts.body = err;
} else {
var results = [];
for (var i = 0; i < result.entries.length; i++) {
connection.getFileInfo(result.entries[i].id, function (err, fileInfo) {
if (err) {
// do error handling here
opts.body = err;
results.push(null);
} else {
results.push(fileInfo);
}
// if done with all requests
if (results.length === result.entries.length) {
// done with everything, results are in results
// process final results here
}
});
}
}
});
});
