How to manage async function after/before loop

Question

How to run an async function after an async loop? A loop does not return a promise which means you cannot use await syntax to manage the execution order. What is a good way to make manage a loop like a promise.

This issue happens in my express routing. I tried to run the res.json after I retrieve data from reids DB with a loop, but res.json will run before the loop if I just put the code after it. Here I used a stupid way. I detect the last iteration of the loop and run my res.json function. I do not think this should be the right way of solving this issue.

app.get('/redisApi/v0.3/users', (req, res) => {
  var users = [];
  redis.scan(0, 'MATCH', 'user:info:*', 'COUNT', 10000).then(key_res => {
    //where loop starts
    for (let i = 0; i < key_res[1].length; i++) {
      redis.hgetall(key_res[1][i]).then(user_res => {
        const newOBJ = Object.assign(user_res);
        users.push(newOBJ);
        //detect last iteration of the loop
        if (i === key_res[1].length - 1) {
          res.status(200).json({
            status: 'success',
            data: {
              users
            }
          });
        }
      });
    }
  });
});

Looking for a way to manage order of execution of loops.

Instead of a loop, use functional programming and `.map()` an array of promises you pass to `Promise.all()` — Patrick Roberts, Sep 24 '19 at 20:45
By the way, `Object.assign(user_res) === user_res`. That does not create a new object reference. — Patrick Roberts, Sep 24 '19 at 20:47
You could move the `res.status...` part outside of the for loop and avoid having to check if it's the last one every time. — solarc, Sep 24 '19 at 20:47
@solarc no you can't. The body of each iteration occurs in an asynchronous callback. "After" the loop does not actually execute sequentially after the iterations, as the question correctly points out. — Patrick Roberts, Sep 24 '19 at 20:49
If you want to run the async processes in parallell but still get the results in the expected order, use `.map()` and `Promise.all()`. If you simply want to run them sequentially, `async-await` syntax would work nice with your existing `for` loop — Lennholm, Sep 24 '19 at 20:49
extract each loop to a separate `async` function, and call them with `await`: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Statements/async_function — Vadim Aidlin, Sep 24 '19 at 20:50
there are many, many questions on Stack Overflow regarding javascript promises inside of loops. which of those questions have you researched and why were their answers not sufficient? — Dan O, Sep 24 '19 at 20:52
@DanO most questions are about async WITHIN the loop. I do not care the order of execution within the loop but after/before it. — Terry Zhou, Sep 24 '19 at 20:55

score 1 · Accepted Answer · answered Sep 24 '19 at 20:56

Here's how I'd write it:

app.get('/redisApi/v0.3/users', async (req, res) => {
  try {
    const key_res = await redis.scan(0, 'MATCH', 'user:info:*', 'COUNT', 10000);
    const users = await Promise.all(key_res[1].map(key => redis.hgetall(key)));

    res.status(200).json({ status: 'success', data: { users } });
  } catch (error) {
    res.status(500).json({ status: 'failure', data: { error } });
  }
});

Notice the use of .map() and Promise.all() in order to asynchronously await each of the requests concurrently.

Don't forget to add error handling. You don't want to leave an HTTP request hanging ~~(pun intended)~~.

Very much appreciated! I found out many similar questions have hints suggesting Promise.all and map function, but they never give actual syntax. — Terry Zhou, Sep 24 '19 at 21:08

score 0 · Answer 2 · answered Sep 24 '19 at 21:06

There are many tools for this problem, here are two:

If you want to fire the async calls one by one, waiting for the last call before making the next call: Use Bluebird.js mapSeries. Provide your key_res[1].length as array to map and the logic of your loop as mapping function.
If you want to fire the async calls all at once but wait for all calls to complete: Simply collect the promises returned by redis.hgetall in a new array and use Promise.all on this array.

How to manage async function after/before loop

2 Answers2