Node.js - Push Recursively into an Array Using Async FS

Question

I am trying to use the fs readdir to create a function that push file names into an array, recursively (including files in sub directories), using default attribute dir as a starting point.

When I tried to console.log the names it worked fine, but I could not figure out how to create an array with all the names in it (I'm not getting the files in the sub directories).

  import { readdir } from 'fs/promises';

   const arr = [];
   const reader = async (dir = `./src`) => {
      try {
         const items = await readdir(dir, { withFileTypes: true });
         items.map(item => item.isDirectory() ? reader(`${dir}/${item.name}`) : arr.push(item.name));
      }
      catch (err) {
         console.log(err);
      }
   };

   reader().then(() => {
      console.log(arr)
   })

Thanks!

Promise.all might help https://stackoverflow.com/a/58362559/13583510 — cmgchess, Jun 26 '22 at 14:52

score 0 · Answer 1 · answered Jul 05 '22 at 01:35

You're not awaiting the call to reader(), and you're also not calling .then() on it.

so execution just continues, console.log(arr) executes with an empty array, and then subsequent calls to reader() actually take place and push to the array.

Fixed version:

const arr = [];
const reader = async (dir = `./src`) => {
    try {
        const items = await readdir(dir, { withFileTypes: true });
        for (const item of items) {
            (item.isDirectory()) {
                await reader(`${dir}/${item.name}`);
            } else {
                arr.push(item.name);
            }
        }
    } catch (err) {
        console.log(err);
    }
};

reader().then(() => {
    console.log(arr)
})

don't `catch` inside the function. let the error bubble up so the consumer can choose what to do with it. if `console.log` is what they want, that is for them to decide. — Mulan, Jul 05 '22 at 03:40

score 0 · Answer 2 · answered Jul 05 '22 at 03:40

You're close but one hint there's a mistake is the use of .map without using its result. .map is constructing an array of promises, each of which need some sort of processing to assemble the final output.

I would recommend using a generator instead of forcing the results into an array. This side-steps the need for .map and allows the consumer to process the resulting files asynchronously as they arrive -

import { readdir } from "fs/promises"
import { join } from "path"

async function* ls(path = ".") {
  for (const dirent of await readdir(path, { withFileTypes: true }))
    if (dirent.isDirectory())
      yield *ls(join(path, dirent.name))
    else
      yield join(path, dirent.name)
}

for await (const f of ls("/foo"))
  console.log("file found:", f)

file1
file2
file3
...

If indeed the consumer wishes to convert the asynchronous stream of elements to a flat array, a generic toArray can be used -

async function toArray(iter) {
  const r = []
  for await (const v of iter)
    r.push(v)
  return r
}

console.log("all files", await toArray(ls("/foo)))

[ file1, file2, file3, ... ]

Node.js - Push Recursively into an Array Using Async FS

2 Answers2