How to use callback or Async in for loop?

Question

I search and read about Asynchronous functions and callbacks. But i was not able to solve my problem for a week. I want to do 'fs readfile' inside loop with order. I try the following but i am not successful.

// on the code below, console.log print the value in random order and 'storedata' is empity.
// my goal is to do readfile in loop orderly and store the value

router.get("/files/readfiles", function(req,res){
  var storedata= []; 
  var filenames= ["file1","file2","file3","file4"]; 

  for (var i=0; i< filenames.length; i++){      
    fs.readFile('views/allfiles/'+ filenames[i] +'.ejs','utf8',function (err, data) {
      if (err) throw err;  
      storedata.push(data);         
      console.log(data);         
    });
    console.log(storedata); // this returns empty array
  });

I also try in another way:

router.get("/files/readfiles", function(req,res){
var filenames= ["file1","file2","file3","file4"];   

filenames.forEach(readfiles);

function readfiles(value) {
  var dataread =  fs.readFile('views/allfiles/'+ value +'.ejs','utf8')

    console.log (dataread);
   }
});

on the above try i get an error of: TypeError [ERR_INVALID_CALLBACK]: Callback must be a function.

I am new to Asynchronous methods any help please.

Possible duplicate of [How do I return the response from an asynchronous call?](https://stackoverflow.com/questions/14220321/how-do-i-return-the-response-from-an-asynchronous-call) — Jared Smith, Nov 20 '18 at 16:39
You will have to either use the sync version or `Promise.all`. Either way you will need to understand what asynchronous *means*, of course `console.log(storedata)` logs an empty array as the IO hasn't completed yet when it runs. If you use a callback you only have access to the data in a callback. — Jared Smith, Nov 20 '18 at 16:39
What exactly is the result you want? Blast through it as fast as possible, or read one file at a time. — Kevin B, Nov 20 '18 at 16:42
@Kevin i want to do fs read file inside loop (read multiple file orderly) then store the value to use it . this is what i want as i wrote in the question. — wzwd, Nov 20 '18 at 16:45
@wzwd Yes, i understand that, but due to the asynchronous nature of reading files, you have two different ways of doing it. You haven't specified which one would be preferred. — Kevin B, Nov 20 '18 at 16:49
@Kevin B if you have solution using Async/await or promise it would be helpful to me. I try the answers on Async/await but they are not working (am not able to fix them). — wzwd, Nov 20 '18 at 17:37

score 0 · Accepted Answer · answered Nov 20 '18 at 16:44

0

If you're using Node v10 or above you can use the fs promises API and async/await.

To read the files in series:

router.get( "/files/readfiles", async function( req, res ) {
  const storedata = [ ]; 
  const filenames = [ "file1", "file2" , "file3", "file4" ]; 

  for (let i = 0; i < filenames.length; i++ ) {      
    const data = await fs.promises.readFile( 'views/allfiles/'+ filenames[i] +'.ejs', { encoding: 'utf8' } ); 
    storedata.push( data );         
  }

  console.log( storedata ); 

} );

or to read them in parallel:

router.get( "/files/readfiles", async function( req, res ) {
  const promises = [ ]; 
  const filenames = [ "file1", "file2", "file3", "file4" ]; 

  for (let i = 0; i < filenames.length; i++ ) {      
    const promise = fs.promises.readFile( 'views/allfiles/'+ filenames[i] +'.ejs', { encoding: 'utf8' } ); 
    promises.push( promise );         
  }

  const storedata = await Promise.all( promises );

  console.log( storedata ); 

} );

answered Nov 20 '18 at 16:44

Paul

139,544
27
275
264

On the 1st method i get: 'parsing error: Unexpected token fs'. I can't modify this line. On the second method i get a result of:` [ Promise { }, Promise { }, Promise { }, Promise { }, Promise { },]` (node:3575) ExperimentalWarning: The fs.promises API is experimental – wzwd Nov 20 '18 at 17:21
@wzwd Sounds like you didn't put the `async` keyword in front of `function`. – Paul Nov 20 '18 at 19:24
The ExperimentalWarning should be there, but it's just printed to stderr and shouldn't affect your program. – Paul Nov 20 '18 at 19:28
Yes i forgot to add Async on the function. It perfectly works. Thanks – wzwd Nov 21 '18 at 10:52

score 0 · Answer 2 · answered Dec 06 '18 at 19:24

You should use ES2017 async/await syntax like this.

router.get("/files/readfiles", async function(req, res){
  var storedata= []; 
  var filenames= ["file1","file2","file3","file4"]; 

  for (var i=0; i< filenames.length; i++){  
    await new Promise((resolve, reject) => {    
        fs.readFile('views/allfiles/'+ filenames[i] +'.ejs','utf8',function (err, data) {
          if (err) return reject( err )
          storedata.push(data);
          resolve();               
        });
    })
    console.log(storedata);
  }
});

iagowp · Answer 3 · 2018-11-20T17:27:10.537

Your second code doesnt provide a callback function, thats your error.

Your first code, you are trying to work with callbacks, so you'll need to follow your code nested into the callback, thats what is called callback hell

It would look like this:

router.get("/files/readfiles", function(req,res){
  var storedata= []; 
  var filenames= ["file1","file2","file3","file4"]; 

  for (let i = 0; i< filenames.length; i++){      
    fs.readFile('views/allfiles/'+ filenames[i] +'.ejs','utf8',function (err, data) {
      if (err) throw err;  
      storedata[i] = data;

      if (storedata.length === filenames.length) {
        console.log(storedata);
        // do stuff you want, like:
        res.send(storedata)
      }            
     });
  });

If you plan on using node 10, I recommend you look at the other answer with Promise.all. Async/await is great

Thanks it is working, I need to read more about callback , Async/await, promise. I am using Node v10.6.0 but i have no idea this time how to use Promise and Async. may be if you have some links to study i will follow it — wzwd, Nov 20 '18 at 17:29

How to use callback or Async in for loop?

3 Answers3