Using data from asynchronous functions

Question

I have question regarding asynchronous functions and how to send something after a function has returned it's result. This is what I am trying to accomplish:

Within the handling of a GET request in Node I read the contents of a folder, returning the files in that folder. Next I want to loop over the stats of each file in that folder, loading only the files created within a certain period and lastly send the data in those files as a response to the request. It looks something like this:

array = []

fs.readdir(path, function(err, items) {
    items.forEach(function(item) {
        fs.stat(path, function(err, stats) {
            if (period check) {
                array.push(data)
            }
        })
    })
}

res.send(array)

This approach ends up sending an empty array, and I've looked into Promises which seem the solution here but I can't get them to work in this scenario. Using fs.statSync instead of fs.stat does work but this greatly reduces the performance, and it feels like it should be doable with Promises but I just don't know how.

Is there anyone with a solution for this?

EDIT: Regarding to the question marked as duplicate, I tried to solve my problem with the answer there first but didn't succeed. My problem has some nested functions and loops and is more complex than the examples given there.

Take a look at the async library, the eachSeries method – yBrodsky Jun 15 '17 at 14:29 — yBrodsky, Jun 15 '17 at 14:29

Patrick Roberts · Accepted Answer · 2017-06-15T14:40:43.967

1

Use this if you prefer a Promise-based approach:

var path = require('path')

fs.readdir(myPath, function(err, items) {
    var array = [];

    Promise.all(items.map(function(item) {
        return new Promise(function(resolve, reject) {
            fs.stat(path.resolve(myPath, item), function(err, stats) {
                if (err) {
                   return reject(err)
                }

                if (/* period check */) {
                  array.push(data)
                }

                resolve()
            })
        })
    })).then(function() {
        res.send(array)
    }).catch(function(error) {
        // error handling
        res.sendStatus(500)
    })
}

edited Jun 15 '17 at 14:40

answered Jun 15 '17 at 14:33

Patrick Roberts

49,224
10
102
153

Alright, I'm finished updating. Let me know if this works for you – Patrick Roberts Jun 15 '17 at 14:45
Thanks, this worked! Is there a specific reason you are using the path package? I just concatenate strings to get the full path. – Evertvdw Jun 20 '17 at 06:33
@Evertvdw It's just generic and ensures cross-platform compatibility for file separators, they're not always forward slashes. – Patrick Roberts Jun 20 '17 at 12:30
It would be best if you surrounded the array push if with a try catch and reject the promise accordingly. Anything bad happening there will be swallowed. – Quentin Roy Sep 16 '17 at 09:45
@QuentinRoy it _is_ swallowed: https://jsfiddle.net/wgtkm3ut/ – Patrick Roberts Sep 16 '17 at 15:37
Yes. You must catch the error and reject the promise with it. – Quentin Roy Sep 16 '17 at 15:53
@QuentinRoy you're missing the point. There is no need for an explicit `try catch` statement, because thrown errors are implicitly caught by the promise chain, and it already has a `.catch()` to swallow errors. There's nothing that needs to be changed – Patrick Roberts Sep 16 '17 at 16:59
When I mean swallow, I mean the error will pass and be lost without being catchable. No, there is no try catch for what is inside `fs.stat` and because `fs.stat` is asynchronous, any error it throws won't be caught by the body of the Promise. – Quentin Roy Sep 16 '17 at 17:08
Try it yourself :) https://jsfiddle.net/1wxomqrp/ – Quentin Roy Sep 16 '17 at 17:10

Quentin Roy · Answer 2 · 2017-06-15T15:04:00.797

Here is what I would suggest.

// This is  a new  API and you might need to use the util.promisify
// npm package if you are using old node versions.
const promisify = require('util').promisify;
const fs = require('fs');

// promisify transforms a callback-based API into a promise-based one.
const readdir = promisify(fs.readdir);
const stat = promisify(fs.stat);

const dataProm = readdir(path)
  .then((items) => {
    // Map each items to a promise on its stat.
    const statProms = items.map(path => fs.stat(path);
    // Wait for all these promises to resolve.
    return Promise.all(statProms);
  })
  // Remove undesirable file stats based on the result
  // of period check.
  .then(stats => stats.filter(stat => periodCheck(stat)));

// dataProm will resolve with your data. You might as well return it
// as is. But if you need to use `res.send`, you can do:
dataProm.then((data) => {
  res.send(data);
}, (err) => {
  // If you go away from the promise chain, you need to handle
  // errors now or you are silently swallowing them.
  res.sendError(err);
});

Here is a link toward the util.promisify package I am referring to. If you are using node v8+, you do not need it. If you do, do not forget to replace require('util').promisify; by require('util.promisify');.

Using data from asynchronous functions

2 Answers2