Nodejs Iterate through array and write to file

Question

I have an array contains around 20K records. Each record contains one encrypted column and rest of them are not. When I iterate through each record using for loop and call the decrypt callback function, the result is coming up later hence the decrypted value are always null. After decrypting I want to write entire row to excel sheet and close the file after every element in array is written to the file. What is the best way to do so?

var myArray = [{name:'ABC',accountNumber:'hsjdhsj%^==='},{name:'BCD',accountNumber:'hsjdhsj%^==='}];
for(var i=0; i< myArray.length; i++){
   myArray[i].decryptAccNumber = decrypt(myArray[i].accountNumber);  // async way to decrypt it 
   res.write(myArray[i]); //Writing to excel file
}
res.close();

Now, the file contains only name, account number but not decrypted account number.

Any advise on how to proceed with such requirement that is mixing of sync & async

---------- Edited ------------------------

var callbackCount = myArray.length;
for(var i=0; i< myArray.length; i++)
{
   decrypt(myArray[i].accountNumber,function(decrypted){  
     myArray[i].decryptAccNumber=decrypted;
     res.write(myArray[i]); //Writing to excel file
     callbackCount--;
     if(callbackCount === 0){
       res.close();
     }
  });
}

This is the solution given at https://gist.github.com/Grety/236c41acf006475f0eee768b64e4a7bc Even, I thought the same solution but personally I did not like this solution as it looks like work around. Is there any other better way to solve this problem other than using temp variable like callbackCounter? This is the exact problem what I want to posted. Thanks to Kyrylo Slatin for posting GIT url with exact problem.

You should make all async. There is no way to right several bytes to files "later". You wait for decryption to finish. And when it is done (not clear since in the gist you didn't provide a callback to `decrypt`) you write to file both decrypted data and those that were obtained synchronously — Kirill Slatin, May 04 '16 at 12:24
write a call back function for this or first decrypt all and then write into file — Asad, May 04 '16 at 12:27
How do we know it is end of row & call close? Is it something like have global variable to keep track count and close them in callback? Is there any example of such? — Senthil Kumar Vaithiyanathan, May 04 '16 at 14:54

score 0 · Answer 1 · edited May 23 '17 at 12:31

0

Pass a callback to your decrypt function. Then you can write your row after the decrypt function finishes. Here is an example

edited May 23 '17 at 12:31

Community

1
1

answered May 04 '16 at 12:29

arodjabel

420
5
14

score 0 · Answer 2 · answered May 04 '16 at 12:32

0

Your problem can easily be solved by using asyn.foreach. It will call your decrypt asynchnously for each and every element in your array and at the end you can have your result./

refer this https://github.com/caolan/async

answered May 04 '16 at 12:32

Nitish Agarwal

692
1
6
16

Asad · Answer 3 · 2016-05-04T13:04:27.197

0

write call back function like this hope it help

  for(var i=0; i< myArray.length; i++)
  {
    decrypt(myArray[i].accountNumber,function(decrypted){  
    myArray[i].decryptAccNumber=decrypted;
   res.write(myArray[i]); //Writing to excel file
    });

  }
  res.close()

and your derypt function would be like this

  decrypt(number,callback){
   var decrypted= // you code of decryption

     callback(decrypted)
  }

edited May 04 '16 at 13:04

answered May 04 '16 at 12:47

Asad

3,070
7
23
61

when should `res.close()` be executed in your code? – Kirill Slatin May 04 '16 at 12:59
after the call back function i edited it @KirillSlatin – Asad May 04 '16 at 13:01
right before all items are decrypted... given that `decrypt` is really async. So `for(` will execute in one js tick, following file closing, and in the next ticks callbacks will be called, attempting to write to just closed file – Kirill Slatin May 04 '16 at 13:03
besides in code `res.close()` is currently placed _inside_ the loop – Kirill Slatin May 04 '16 at 13:04
thanx for corrention @KirillSlatin . now its okay ? – Asad May 04 '16 at 13:04
well im also now in nodejs so .. :) – Asad May 04 '16 at 13:05
no it is not, as I mentioned in my second statement. If `decrypt` is really async (will wait for some io operation) and callbacks will be called much later then the cycle ends. The code you proposed for `decrypt` is not really async. You just use a callback, but all code is executed synchronously – Kirill Slatin May 04 '16 at 13:06
can u make it correct . ? edit my code ? i also want to learn it – Asad May 04 '16 at 13:07
Let us [continue this discussion in chat](http://chat.stackoverflow.com/rooms/111003/discussion-between-asad-and-kirill-slatin). – Asad May 04 '16 at 13:08

Nodejs Iterate through array and write to file

3 Answers3