2

I am aware of io-q, the library that does async IO resulting in promises. But I'm looking for a simple example of using the Q library to recursively traverse a directory structure, where the final result is a list of all the files in all the directories starting in the folder provided to some function, but flattened in the process to a single array of file names.

Is there an example of this out there? Or perhaps there's an example that isn't recursive, which is fine. I'm guessing this is pretty simple, but this is my first exposure to both async/promises.

lucidquiet
  • 6,124
  • 7
  • 51
  • 88

3 Answers3

4

I found this gist which does what you want and is easy to promisify:

var Q = require('q'),
    fs = require('fs'),
    p = require('path');
function readDir(path) {
    return Q.nfcall(fs.lstat, path).then(function(stat) {
        if (stat.isDirectory()) {
            return Q.nfcall(fs.readdir, path).then(function(files) {
                return Q.all(files
                // .map(p.join.bind(p, path)).map(readDir)
                .map(function(file) {
                    return readDir(p.join(path, file));
                })
                ).then(
                // Function.apply.bind(Array.prototype.concat, [])
                function(results) {
                    return [].concat.apply([], results);
                });
            });
        } else {
            return [path];
        }
    });
}

It uses nfcall to get promises for the filesystem API and Q.all to wait for all subdirectory results before concatenating them.

Bergi
  • 630,263
  • 148
  • 957
  • 1,375
  • Did you test this ? ;) – Esailija Oct 14 '13 at 15:08
  • @Esailija: Ah, I must have forgotten something. Does it work now (still cannot test)? – Bergi Oct 14 '13 at 23:41
  • Nope, still somehow passes files to readdir which will cause not a directory exception. I made this example for my promise lib: https://github.com/petkaantonov/bluebird/wiki/Snippets#reading-directory-and-sub-directory-contents-recursively it could be translated to Q by changing map/reduce – Esailija Oct 15 '13 at 04:14
1

Well, this is the solution I finally arrived at (CoffeeScript). I'm not a Node or Q expert so I guess this will do for now. My solution actually flattens the list, and the way to remove the directories from the output is to remove the .then() after the read(f).

Q = require('q')
fs = require('fs')
_ = require("lodash")

isFile = (name) ->
    fs.statSync(name).isFile()

withDir = (dir) ->
    (files) -> _.map(files, (f) -> "#{dir}/#{f}")

read = (dir) ->
    Q.nfcall(fs.readdir, dir)
        .then(withDir dir)
        .then((files) -> Q.all(toPromises files))

toPromises = (files) ->
    for f in files
        if isFile f then Q(f) else read(f).then((m) -> m.concat("d " + f))

read("my-root-dir")
    .then((files) -> _.flatten(files))
    .then((r) -> console.log r)
lucidquiet
  • 6,124
  • 7
  • 51
  • 88
  • 1
    Well you are using `statSync` which makes this code pointless: you might as well *Sync everything and make it even easier. You won't be needing promises either :p. – Esailija Oct 15 '13 at 04:18
  • @Esailija yeah -- you are right. This is mostly for self-teaching, but that was going to be my next step: remove the statSync so that too is async. I'll update when I have that working. – lucidquiet Oct 15 '13 at 15:20
1

The listTree function from Q-IO does exactly what you're looking for, so you could take a look at the implementation.

Community
  • 1
  • 1
Stuart K
  • 3,212
  • 4
  • 22
  • 27