How to synchronize an async map function in javascript

Question

I have a async map function but want it to execute synchronously because I need the output of the first statement to be used within the same loop. But the map runs asynchronously even with await statements, can you please help understand why this happens.

My use case if to insert a record into mongodb if not present and update it if present in a loop. The data exists in the db but find fails within the loop but works outside.

My code:

        const doSomethingAsync = () => {
            return new Promise(resolve => {
                setTimeout(() => {
                    resolve(Date.now());
                }, 1000);
            });
        };

        await Promise.all(
            modelVarients.map(async varient => {
                console.log(`varient: ${varient._id}`);
                console.log('1');
                const onlineDevice = await Device.findOne({
                    model: varient._id,
                });
                console.log('2');
                await doSomethingAsync();
                console.log('3');
                await doSomethingAsync();
                console.log(JSON.stringify(onlineDevice));
                await doSomethingAsync();
                console.log('4');
                return varient;
            })
        );

Logs I get:

varient: 8 pro
1
varient: note
1
varient: iphone x
1
2
2
2
3
3
3
null
null
null
4
4
4

But what I expect to get:

varient: 8 pro
1
2
3
<actual response from db for 8 pro>
4
varient: note
1
2
3
<actual response from db for note>
4
varient: iphone x
1
2
3
<actual response from db for iphone x>
4

Does this answer your question? [Is Node.js native Promise.all processing in parallel or sequentially?](https://stackoverflow.com/questions/30823653/is-node-js-native-promise-all-processing-in-parallel-or-sequentially) — Edward Romero, Sep 05 '20 at 13:42
use await modeVariants.reduce( async (...) => ) instead of map — olkivan, Sep 05 '20 at 13:48
@EdwardRomero thanks for pointing me to a similar question, yes it helped me learn about reduce and how to chain promises, the answer below from sashee using async/await is also very helpful — Sai Pravesh, Sep 05 '20 at 15:22

score 4 · Accepted Answer · answered Sep 05 '20 at 13:47

4

The modelVarients.map(async () => >...) converts all the elements into Promises, which means they all start executing. Then the Promise.all() collects them and waits for all of them, that's why you can use this structure to wait for the map to finish. This is parallel processing.

What you need is sequential processing, which you can do with a reduce, like this:

await modelVarients.reduce(async (memo, varient) => {
    await memo;
    // all the other things
}, Promise.resolve())

reduce is similar to map in a sense that it creates a Promise for all the elements in the array, but there is a current value that is passed from one element to the other. In this case, it is the Promise.resolve() for the first one, the result of the first for the second, and so on. With the await memo you can wait for the previous result.

With reduce, the last element will wait for the previous one, which waits for the previous one, and so on, so there is no need for a Promise.all.

I've written articles about how map and reduce works with async functions, they will help you see the big picture.

answered Sep 05 '20 at 13:47

Tamás Sallai

3,195
1
14
25

Using `reduce` with an `async function` as the callback is a bad idea, it's inefficient and too easy to get completely wrong. – Bergi Sep 05 '20 at 14:07
@Bergi can you elaborate why you think this is a bad idea, when I looked into reduce even the official docs has a simialar example https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Array/Reduce – Sai Pravesh Sep 05 '20 at 15:16
@Sai That's using `then` not `await`, and is chaining (few) different functions from an array instead of the same function on (many) items in the array. – Bergi Sep 05 '20 at 15:21
In general, just use a `for … of` loop and `await` - much simpler, shorter, and faster. – Bergi Sep 05 '20 at 15:21
es-lint discourages for ... of when this error: "iterators/generators require regenerator-runtime, which is too heavyweight for this guide to allow them. Separately, loops should be avoided in favor of array iterations." https://eslint.org/docs/rules/no-restricted-syntax any thoughts? – Sai Pravesh Sep 05 '20 at 15:53
@Sai My thoughts on that? es-lint (or rather, that specific configuration) is just wrong. The array iteration methods don't work with `await`, and `for … of` is idiomatic and well-supported in modern environments. You can of course use `for (var i=0; i – Bergi Sep 07 '20 at 02:10

score 0 · Answer 2 · answered Sep 05 '20 at 14:03

0

looks like You should chain Your promises, not running then in parallel with Promise.all

answered Sep 05 '20 at 14:03

fedeghe

1,243
13
22

How to synchronize an async map function in javascript

2 Answers2

Linked