Read files in series with fs.readFile in Node?

Question

I have an arbitrary number file paths stored in an array as strings.

I need to read these files in series (i.e. finish reading one file before starting reading the next one), and output the file length after each file read. And after finish reading all files, print 'done'.

I'm only allowed to use fs.readFile and native Javascript. No fs.readFileSync or any other module is allowed. Also, I'm on Node v6, so I can't use async/await.

Is there any way to implement the functionality, following the constraints?

yes, you need to use the callbacks of one read to know when to begin the next ... Promises (probably need a library for that in such an ancient node) will make it much easier — Jaromanda X, Dec 18 '18 at 12:04
@JaromandaX Yea I figure that would be the idea. But how do I implement it over an array of arbitrary size? — delsin, Dec 18 '18 at 12:06
I like using array reduce with promises for such tasks - but that is by no means the only method — Jaromanda X, Dec 18 '18 at 12:07
https://medium.com/@ajmeyghani/writing-asynchronous-programs-in-javascript-9a292570b2a6 — VinoPravin, Dec 18 '18 at 12:10
The constraint on `fs.readFileSync` seems a bit arbitrary for a real-world problem; is this homework? — Paul, Dec 18 '18 at 12:12
@Paul Nah, but close. It's part of a training program at my new job. I'm fairly new to Node. — delsin, Dec 18 '18 at 12:14
Ok, so as others have pointed out, you *can't* make it synchronous. Are you sure that's the requirement, or do you just mean the files have to be read in order / in series? — Paul, Dec 18 '18 at 12:15
@Paul Maybe I should have worded it a bit better. Yes, I meant the files needed to be read in series. — delsin, Dec 18 '18 at 12:19

kucherenkovova · Accepted Answer · 2021-11-29T11:46:28.940

4

You don't need any fancy stuff like promises, async/await, generators to achieve that.

function readAllFiles(list) {
  if(list.length == 0) return
  console.log('reading ', list[0])
  return fs.readFile(list[0], function (err, file) {
    if (err) console.error(err); // report the error and continue
    console.log('list[0] length:', file.length); // report the file length
    readAllFiles(list.slice(1, list.length))
  })
}

var l = ['1', 'foo', 'bar']

readAllFiles(l)

edited Nov 29 '21 at 11:46

answered Dec 18 '18 at 12:17

kucherenkovova

694
9
22

btw, when using recursion one should always remember that stack is not infinite. If you going to process long lists this way it can be useful to wrap nested `readAllFiles` in `setTimeout`/`setImmediate`. More details: https://stackoverflow.com/questions/20936486/node-js-maximum-call-stack-size-exceeded – Alex Povar Dec 18 '18 at 13:58
1

@AlexPovar this can't lead to stack overflow errors since `readAllFiles` doesn't invoke itself. It's called by a callback that we pass to `fs.readFile` function. – kucherenkovova Dec 18 '18 at 14:52

score 0 · Answer 2 · answered Dec 18 '18 at 12:11

Try use generators. In example code replace acync function to reading files.

// emulate async action
const stt = (resolve, str) => {
  setTimeout(
    () => {
      console.log(str);
      resolve(str);
    },
    2000
  );
}
// Create array of functions that return Promise
const arrFunc = ['foo', 'bar', 'baz', 'qux']
  .reduce(
    (acc, str) => {
      acc.push(() => {
        return new Promise(resolve => {
          stt(resolve, str);
        });
      });
      return acc;
    }, []
  );

function* generator() {
  for (let func of arrFunc) {
    yield func();
  }
}

const iterator = generator();

function quasiCo(generator, value) {
  const next = generator.next();
  console.log(next);
  if (!next.done) {
    next.value.then(res => {
      quasiCo(generator, res);
    });
  }
}

quasiCo(iterator);

score -1 · Answer 3 · answered Dec 18 '18 at 12:10

Read files synchronously with fs.readFile in Node?

No.

Asynchronous functions are asynchronous.

They can't be turned into synchronous functions.

Even if you could use await (which you've ruled out), it would still be asynchronous just with syntax that let you write code in a synchronous style within a wider context that is still asynchronous.

Much of JavaScript is asynchronous. Embrace it. Lean to use Promises. Don't try to fight it.

I need to read these files in sequence (i.e. finish reading one file before starting reading the next one), and output the file length after each file read. And after finish reading all files, print 'done'.

Write a recursive function that iterates along an array. When you've read the file, either increment the iterator and recurse or print done depending on if you are at the end of the array or now.

Read files in series with fs.readFile in Node?

3 Answers3