How to wait for multiple asynchronous calls from forEach loop?

Question

Trying to make call to multiple asynchronous function, and due to which getting result as undefined.

Tried async.waterfall, however not able to make it work.

Code:

const pendingData = [];

async.waterfall([
    function (callback) {
      WaitForApproval.find({}, (err,result) => {
        callback(null,result);
      });
    },
    function (result, callback) {
      result.forEach(data => {
        User.findOne({_id: data.uploadedBy}, (err,name) => {
          let total = {
            id: data._id,
            name: name.name,
            subject: data.subject,
            uploadOn: data.uploadedAt
          };
          pendingData.push(total);
        });
      });
      callback(null,'done');
    }
  ], function (err,result) {
    if(result === 'done') {   
      console.log(pendingData); // it is giving empty result.
    }
  });

How to wait for asynchronous function?

Just change `result.forEach` to `async.series` or `async.parallel` — Cody Ｇ, Sep 13 '18 at 18:42
Possible duplicate of [How to wait for multiple asynchronous calls from for loop?](https://stackoverflow.com/questions/52276621/how-to-wait-for-multiple-asynchronous-calls-from-for-loop) — NAVIN, Sep 13 '18 at 19:04

AlexanderGriffin · Accepted Answer · 2018-09-13T18:47:57.583

2

The issue you are having is that you are async functions within a non-async forEach loop.

You have a few options here:

Make the mongoDB call recursively - wrap this query in a function that calls itself after the query returns.
Read about mongoDB batch operations - https://docs.mongodb.com/manual/reference/method/Bulk/
Use an async/await pattern with each call by declaring the callback function within the async waterfall as 'async' and then using 'await' inside of the function for each query. Out of the box, forEach is not async. If you want to still use forEach, you can either re-write it async (See below) or use a regular for loop:

async function asyncForEach(array, callback) {
  for (let index = 0; index < array.length; index++) {
    await callback(array[index], index, array)
  }
}

** There are additional ways to solve this beyond what is posted here, but here's a few things that will work if implemented correctly.

edited Sep 13 '18 at 18:47

answered Sep 13 '18 at 18:30

AlexanderGriffin

515
2
13

1

This will fire the call back after the first `User.findOne` finishes. That's too early. Perhaps you could test for the length of `pendingData`: `if (pendingData.length === result.length) callback(null, pendingData)` – Mark Sep 13 '18 at 18:32
It is giving an error as I am using `forEach` for which `callback` will use outside the loop. – Rupesh Yadav Sep 13 '18 at 18:35
1

@MarkMeyer you are correct! I did not realize there was a forEach being called here - I apologize; I quickly responded to this because I've just dealt with the same issue at work :) – AlexanderGriffin Sep 13 '18 at 18:37
@MarkMeyer, This trick is working. But surely there would also be the other way to solve this. – Rupesh Yadav Sep 13 '18 at 18:39

score 2 · Answer 2 · answered Sep 13 '18 at 18:41

I would suggest you add the call for callback(null,'done'); immediately pendingData.push(total);

you are right, the async nature is making things hard for you now, but suppose you used Promises and chain them together, that would save you a lot of trouble.

once a time I had a similar problem of asynchronous code running out of order I wanted so I made a little Tweak using a custom promise function(I mead up) and call it order..such Idea can solve your problem if you could apply it to your code properly https://github.com/lalosh/Ideas/blob/master/promiseOrder.js

Your snippet is great but there are a few things I think that could improve it: - Remove the console log integers, they are extraneous - If you use async/await architecture along with promisified functions, you can avoid the .then() callbacks — AlexanderGriffin, Sep 13 '18 at 18:52

How to wait for multiple asynchronous calls from forEach loop?

2 Answers2