Parallel operations with Promise.all?

Question

I'm led to believe that Promise.all executes all the functions you pass it in parallel and doesn't care what order the returned promises finish.

But when I write this test code:

    function Promise1(){
        return new Promise(function(resolve, reject){
            for(let i = 0; i < 10; i++){
                console.log("Done Err!");
            }
            resolve(true)
        })
    }
    
    function Promise2(){
        return new Promise(function(resolve, reject){
            for(let i = 0; i < 10; i++){
                console.log("Done True!");
            }
            resolve(true)
        })
    }
    
    Promise.all([ 
        Promise1(),
        Promise2()
    ])
    .then(function(){
        console.log("All Done!")
    })

The result I get is this

Done Err!
Done Err!
Done Err!
Done Err!
Done Err!
Done Err!
Done Err!
Done Err!
Done Err!
Done Err!
Done True!
Done True!
Done True!
Done True!
Done True!
Done True!
Done True!
Done True!
Done True!
Done True!
Done!

But if they're running in parallel wouldn't I expect them to be executing at the same time and give me a result like this?

Done Err!
Done True!
Done Err!
Done True!
Done Err!
Done True!
Done Err!
Done True!
Etc. Etc.?

Or am I missing something in the way I'm doing it?

So if one of the promises were to go and get something from a database and another were to, say, get data from a web API it would do that asynchronously, but anything I told it to print to the screen would be done in the order I specified in the array? — Steve, Mar 13 '17 at 20:28
It has nothing to do with `Promise.all`. When you create a promise, the function you pass in is evaluated immediately, so the body of `Promise1` is run before you ever call `Promise2` or `Promise.all`. Javascript is not multi-threaded, so your `for` loops will never overlap like that. — SimpleJ, Mar 13 '17 at 20:32
[Here's](http://codepen.io/FullR/pen/WpOgBp?editors=0010) an example of how `Promise.all` works. The promises start in the order they're created, but they run in parallel. — SimpleJ, Mar 13 '17 at 20:36
Thanks for that, I think I understand, but I'm not 100%. Is this caused by the fact that setTimeout has a callback function so it can go off and do other things in the mean time? — Steve, Mar 13 '17 at 20:44
Possible duplicate of [Is Node.js native Promise.all processing in parallel or sequentially?](http://stackoverflow.com/q/30823653/1048572). Notice that a promise does not make a synchronous loop "parallel" or "asynchronous" by itself, it only helps to represent an already asynchronous thing (like database lookups, web api calls or timeouts). — Bergi, Mar 13 '17 at 21:58

score 18 · Accepted Answer · edited Apr 27 '21 at 17:31

18

It's because your Promises are blocking and synchronous! Try something with a timeout instead of a synchronous loop:

    function randomResolve(name) {
      return new Promise(resolve => setTimeout(() => {
        console.log(name);
        resolve();
      }, 100 * Math.random()));
    }
    
    Promise.all([ 
        randomResolve(1),
        randomResolve(2),
        randomResolve(3),
        randomResolve(4),
    ])
    .then(function(){
        console.log("All Done!")
    })

edited Apr 27 '21 at 17:31

logi-kal

7,107
6
31
43

answered Mar 13 '17 at 21:25

Johannes Merz

3,252
17
33

1

Sorry for my ignorance, I understand this for single core CPU. If I had a dual-core CPU, I expected at least two tasks to run parallelly. Like: iterating 2 arrays simultaneously as in the example. – Shihab Jan 24 '19 at 12:36
1

javascript is single threaded. Your cpu count doesnt matter, and even then a synchronous operation can only run in one cpu. – Johannes Merz Jan 24 '19 at 13:19

score 6 · Answer 2 · answered Aug 21 '19 at 17:29

6

I'd suggest to use it like this:

const [
    res1,
    res2
] = await Promise.all([
    asyncCall1(),
    asyncCall1(),
]);

answered Aug 21 '19 at 17:29

k06a

17,755
10
70
110

2

Why is that ? whats the advantage? – O'Dane Brissett Apr 01 '20 at 20:36

score 2 · Answer 3 · edited Jan 13 '21 at 23:02

To build on what Johannes Merz started, I propose this code to clarify that things are happening in parallel.

JS is single-threaded, but Node.js has plenty of tools for launching additional threads explicitly and implicitly. Promises expose more functionality we often need without having to launch new threads or processes explicitly. Promise.all() is such an example, but you need to be comfortable with Promises to use it without creating serious headaches for yourself, such as Promise Scope memory leaks.

    function randomResolve(name,t) {
      return new Promise(resolve => setTimeout(() => {
        console.log({ name, t });
        resolve({ name, t });
      }, t));
    }
    
    (() => {
        // Get epoch time before starting so we can confirm the execution time reflects our slowest timeout
        let start = new Date().valueOf(); 
    
        Promise.all([ 
            randomResolve(1, 1000 * Math.random()),
            randomResolve(2, 1000 * Math.random()),
            randomResolve(3, 1000 * Math.random()),
            randomResolve(4, 1000 * Math.random()),
        ])
        .then(function( res ){
            console.info( res );
            console.log("All Done!", parseInt(new Date().valueOf() - start) );
        })
    })();

This pattern takes an array of inputs and uses array.map() to get back an array of launched promises which will be parallel processed as above. Notice there is NO use of async/await here.

    function randomResolve(name,t) {
      return new Promise(resolve => setTimeout(() => {
        console.log({ name, t });
        resolve({ name, t });
      }, t));
    }
    
    (() => {
        // Get epoch time before starting so we can confirm the execution time reflects our slowest timeout
        let start = new Date().valueOf(),
            vals = [ 
                [1, 1000 * Math.random()],
                [2, 1000 * Math.random()], 
                [3, 1000 * Math.random()],
                [4, 1000 * Math.random()]
            ];
    
        Promise.all( vals.map( v => { return randomResolve(v[0], v[1] ); } ) )
        .then(function( res ){
            console.info( res );
            console.log("All Done!", parseInt(new Date().valueOf() - start) );
        })
    })();

This version does implement async/await.

    function randomResolve(name,t) {
      return new Promise(resolve => setTimeout(() => {
        console.log({ name, t });
        resolve({ name, t });
      }, t));
    }

    (async () => {
        // Get epoch time before starting so we can confirm the execution time reflects our slowest timeout
        let start = new Date().valueOf(),
            vals = [ 
                [1, 1000 * Math.random()],
                [2, 1000 * Math.random()], 
                [3, 1000 * Math.random()],
                [4, 1000 * Math.random()]
            ];
    
        let res = await Promise.all( vals.map( async v => { return await randomResolve( v[0], v[1] ); } ) );
        // await the Promise.aall() call instead of using .then() afterwards with another closure then
        //     forEach v in vals, start and await a Promise from randomResolve() then return the result to map
    
        console.info( res );
        console.log("All Done!", parseInt(new Date().valueOf() - start) );
    
    })();

score 1 · Answer 4 · answered Apr 19 '17 at 19:08

1

A non-Async body is executed serially The moment you reach a Async call within your body (Ex. Hit a URL), Other Promises in the array will start executing.

answered Apr 19 '17 at 19:08

Sanket Berde

6,555
4
35
39

ZecKa · Answer 5 · 2020-01-15T10:56:39.450

Maybe the solution would be to use worker-farm, I wouldn't know how to explain it because I'm new to NodeJS, but here's an interesting article about it:

https://blog.logrocket.com/node-js-multithreading-what-are-worker-threads-and-why-do-they-matter-48ab102f8b10/

In this case, here's how you could use worker-farm
(I don't have a good grasp of the subject at the moment so don't hesitate to correct me.)

First, install "worker-farm"

npm install --save worker-farm

then:

// index.js
const workerFarm = require("worker-farm");
const service = workerFarm(require.resolve("./service"));
function Promise1() {
  return new Promise(function(resolve, reject) {
    service("promise1", (err, output) => {
      resolve(true);
    });
  });
}

function Promise2() {
  return new Promise(function(resolve, reject) {
    service("promise2", (err, output) => {
      resolve(true);
    });
  });
}

Promise.all([Promise1(), Promise2()]).then(function() {
  console.log("All Done!");
});

In service.js create a function that takes a lot of execution time, execute the callback once the execution is finished.

// service.js
const blocker = (input, callback) => {
  // Number loop turns
  // Adjust this number depending on your CPU
  const nbTurn = 1000000000;
  // How many log to display during the loop
  const nbReminder = 4;
  let i;
  for (i = 0; i <= nbTurn; i++) {
    const remainder = (i % Math.ceil(nbTurn / nbReminder)) / 100;
    if (remainder === 0) {
      console.log(input, i);
    }
  }
  console.log(input + "end", i);
  callback(null, nbTurn);
};
module.exports = blocker;

EDIT: I found another solution: napajs (working example on codesandbox)

Parallel operations with Promise.all?

5 Answers5

Linked