Why is await not waiting in fs

Question

I was trying to parse csv file and according to the information fetched, I am making an api call with it.

Here is the code:

let booklist = [];
let updatedBook;
fs.createReadStream("Book1.csv")
.pipe(csv())
.on('data', async function(data){
  console.log("data");
  updatedBook = await apiCall(data.title, data.author, booklist);
})
console.log(updatedBook);
return updatedBook;

The expected behavior here is print data, execute apiCall() function, get the next data in CSV file, print data, execute apiCall() and so on.

however, it does not await for the apiCall to be finished and reads the next data in the CSV file. So, at last, it just returns undefined updatedBook.

It returns proper updatedBook if I sleep for 10 seconds and wait for all the apiCall to be finished but I don't think this is the right way to do.

What am I missing here? is await not supposed to work in this particular function?

Your answer will be very much appreciated! Thanks!!

You are logging `updatedBook` before the `data` event starts.. that streaming starts asynchronously... — Mat J, Aug 21 '19 at 06:03
Possible duplicate of [callback to handle completion of pipe](https://stackoverflow.com/questions/11447872/callback-to-handle-completion-of-pipe) and some others: [1](https://stackoverflow.com/questions/33599688/how-to-use-es8-async-await-with-streams) [2](https://stackoverflow.com/questions/53210627/waiting-until-readstream-has-ended-and-returning-the-data) [3](https://stackoverflow.com/questions/37837132/how-to-wait-for-a-stream-to-finish-piping-nodejs) [4](https://stackoverflow.com/questions/37229725/wait-for-all-streams-to-finish-stream-a-directory-of-files) — ggorlen, Aug 21 '19 at 06:10
@MatJ ohh I get it - is there a way to return the updatedBook after it has finished all the apiCall()? — jinjineos, Aug 21 '19 at 06:13
@jinjineos, see the answer below or [this answer](https://stackoverflow.com/a/53261553/219933) provides another way.. — Mat J, Aug 21 '19 at 06:42

score 1 · Answer 1 · edited Aug 22 '19 at 02:55

Just "wait until" reading csv file finish, try to catch 'finish' event, then wrap all of it into a Promise, in the Promise, we update call update books.

(You can cover reading file error case with error event)

const yourFunctionName = async () => {
  let booklist = [];
  let updatedBooks = []; // I think you have too many books to update.

  await new Promise((resolve) => {
    fs.createReadStream("Book1.csv")
      .pipe(csv())
      .on('data', async function (data) {
        console.log("data");
        const updatedBook = await apiCall(data.title, data.author, booklist);
        updatedBooks.push(updatedBook); //
      })
      .on('finish', () => { // fires when no more data will be provided.
        resolve(); // I'm done, go to `console.log(updatedBooks);`
      });
  });
  console.log(updatedBooks);
  return updatedBooks;
}

As for WHY this is happening: await, as you know, works in async function. This means that the await only applies to the context in which it is called. Basically, your code sees "async" function- it already marks it as asynchronuos and sends it to event loop. Therfore, The "await" would only block execution inside the function where it si found, but main code would still run. This is why you have to do data trasformation related to the "awaited" functionality inside the same async function.

I think the problem now is that fs is reading the subsequent data without waiting for the `apicall` to be finished. So after reading all the data, it will actually just return the empty updatedBooks because none of the apiCall has been processed by that time. — jinjineos, Aug 22 '19 at 05:56
oh, you're right. i'll edit the answer as well, but what you should do is return the awaited promise, and resolve with the parameter you want to return from yourFunctionName, e.g updatedBooks. My edit should be accepted soon and you could see it in answer — Gibor, Aug 22 '19 at 06:44
But, beware that `await` will cause this function to also return a promise, so u'll have to wait for it outside too — Gibor, Aug 22 '19 at 06:46

Why is await not waiting in fs

1 Answers1