Waiting for async function to complete inside map

Question

consider the following simple code code:

await Promise.all(arrayOfObjects.map(async (obj) => {
    return await someAsyncFunctionOnObj(obj);
}));

My problem is, arrayOfObjects, and someAsyncFunctionOnObj, take too much memory while executing, because the loop doesn't wait for the execution to finish, instead it calls someAsyncFunctionOnObj(obj), on each one, and waits till all are resolved, not necessary in order, this causes OOM crash. Iv'e tried using a recursive async function, which does solve the order problem, but still causes OOM crash.

The flow I want to achieve is a synchronous loop, meaning

await someAsyncFunctionOnObj(obj1); // wait for it to finish
await someAsyncFunctionOnObj(obj2); // wait for it to finish
...

Any suggestions on how to correctly implement it?

First of all, it should be `return await someAsyncFunctionOnObj(obj);` But you really don't need async/await if you use `Promise.all`. Or just `await Promise.all(arrayOfObjects.map(someAsyncFunctionOnObj));` — dfsq, Oct 18 '17 at 11:12
Why do you use Promise.all at all when you rather want to chain them? — Rob, Oct 18 '17 at 11:16
@Robert that was exactly my question, what is the best way to chain them. — vrachlin, Oct 18 '17 at 11:23
You probably want the example from [here](https://stackoverflow.com/a/20105079/1195949), just replace RSVP with Promise in the first code example. It explains it pretty well too — tocallaghan, Oct 18 '17 at 11:32

Wing · Accepted Answer · 2018-03-09T17:23:40.470

Solution

async function queueAsyncFns(fns) {
  const values = [];

  await fns.reduce((previous, current, index, array) => {
    const thenable = index === 1 ? previous() : previous;
    return thenable.then(value => {
      values.push(value);
      return index === array.length - 1 ? current().then(value => values.push(value)) : current();
    });
  });

  return values;
}

Example

const anArray = [1, 2, 3];
const doSomething = async (id) => await fetch(`https://jsonplaceholder.typicode.com/users/${id}`).then(res => res.json());

queueAsyncFns(anArray.map((val) => () => doSomething(val))).then((val) => console.log(val));

The above function should solve your issue. Here's a brief overview of what it does:

queueAsyncFns accepts an array of functions that returns the result of calling an async function. This array is reduced by calling each function and returning the Promise to the next call of the reducer. With each iteration the value of the async call is accumulated into an array called values which is returned after all items have been iterated through.

The accurate behaviour of the function can be determined visually by looking at the waterfall graph when running the example. You can see each network call is only made after the previous one has been completed.

Thank you for that explanation, this better answers my question! — vrachlin, Oct 18 '17 at 18:23

Faly · Answer 2 · 2017-10-18T11:30:58.987

1

If you want to wait for someAsyncFunctionOnObj(obj1) to finish before doing the same but with the next object (obj2, obj3, ...), I think you have to chain your promises:

var promises = arrayOfObjects.map(obj => someAsyncFunctionOnObj(obj));
await promises.reduce((m, o) => m.then(() => o), Promise.resolve());

edited Oct 18 '17 at 11:30

answered Oct 18 '17 at 11:20

Faly

13,291
2
19
37

Could you explain the second line? – vrachlin Oct 18 '17 at 11:24
It takes all the promises in var promises and waits for them, one after the other, by the use of array reduce (array reduce use m as memory of the previous promise so we can do .then on m and return the next promise o, o becoming the new value of m and so on) – Faly Oct 18 '17 at 11:30

score 0 · Answer 3 · answered Oct 18 '17 at 11:49

(async function() {
  async function executeSequentially() {
    const tasks = [1,2]
    const total = []
    for (const fn of tasks) {
      const res = await fetch(endpoint);
      const res2 = await res.json();
      console.log(res2[0]);
      total.push(...res2);
    }

    return total;
  }

  const res = await executeSequentially();
  console.log(res);
})();

Waiting for async function to complete inside map

3 Answers3

Solution

Linked