0

I'am trying to invoke the same function parallely, but with different arguements. I used Promise.all, but that doesn't seem to run the tasks parallely. I tried using bluebird, but still it seems that the executions are happening sequentially only. PFB the snippet of code and logs.

let records = await getRecords(query);
    if (_.size(records) > 0) {
       bluebird.map(records, function (record) {
            return prepareFileContent(record.MESSAGE_PAYLOAD);
        }, { concurrency: records.length }).then(function (data) {
            finalData = data;
            console.log("done");
        });            
    }


 export async function prepareFileContent(payload : string) : Promise<string>{
return new Promise<string>(function(resolve,reject){
    try{
        console.log("content generation starts");
        //logic goes here
        console.log("content generation ends");
       resolve(details);          
   }
    catch(err)
    {
        log.error("Error in parsing the payload:", err);
        reject(err);
    }
});`

Logs look something like this which shows that they are executed sequentially and not parallely. (from the time here, each one takes 4 seconds for execution)

2018-04-16T08:47:53.095Z    content generation starts
2018-04-16T08:47:57.819Z    content generation ends
2018-04-16T08:47:57.820Z    content generation starts
2018-04-16T08:48:02.253Z    content generation ends
2018-04-16T08:48:02.254Z    content generation starts
2018-04-16T08:48:06.718Z    content generation ends
2018-04-16T08:48:06.718Z    content generation starts
2018-04-16T08:48:11.163Z    content generation ends
2018-04-16T08:48:11.163Z    content generation starts
2018-04-16T08:48:15.573Z    content generation ends
2018-04-16T08:48:15.574Z    content generation starts

Can someone help me out on how to achieve the same parallely and what am I missing here?

Abirami
  • 223
  • 6
  • 21
  • Please post your actual code where you are really doing some asynchronous work. – Bergi Apr 16 '18 at 21:00
  • Passing `concurrency: records.length` as an option should not be necessary. Also use `await bluebird.map(…)` instead of calling `.then()`. And you [don't need to make the `prepareFileContent` function `async`](https://stackoverflow.com/questions/47880415/what-is-the-benefit-of-prepending-async-to-a-function-that-returns-a-promise). – Bergi Apr 16 '18 at 21:03

1 Answers1

2

First off, node.js Javascript is single threaded. So, no two pieces of Javascript are ever truly run in parallel. When people speak of things running parallel, that only really applies to asynchronous operations that have a native code component such as networking operations, file operations, etc...

It appears that you're operating under an assumption that promises and functions like Bluebird's Promise.map() enable parallel operation. That is only true if the underlying operations you're monitoring with promises (your prepareFileContent() function in your example) are actually capable of running by themselves outside of the Javascript interpreter. But, the code from your function prepareFileContent() that you show us is just Javascript so it can't ever run in parallel with anything else. Remember, node.js runs your Javascript single threaded so it can't run two pieces of Javascript at the same time, ever.

So, your output is exactly as expected. bluebird.map() iterates through the array, calling your callback on each item in the array and collecting a promise from each function call. Then, it waits for all the promises to be done and collects all the resolved results into an array for you.

But, each of your callbacks is synchronous. They don't have any asynchronous part to them so all your code ends up running synchronously. Nothing runs in parallel.

jfriend00
  • 683,504
  • 96
  • 985
  • 979
  • Thanks for the explanation. Clears up so many things. So there is no way to achieve the above requirement of running them in parallel right? – Abirami Apr 18 '18 at 01:30
  • @Abirami - Not when you don't have real asynchronous operations. If the work being done was really asynchronous (like doing a database lookup or read/writing from a file or requesting data from another server), then those could all be done in parallel. – jfriend00 Apr 18 '18 at 01:31
  • @Abirami In JS you can run sychronous tasks in parallel by assigning them to Web Workers at the client side by the [Workers API](https://developer.mozilla.org/en-US/docs/Web/API/Worker) and to [webworker-threads](https://www.npmjs.com/package/webworker-threads) at the server side but these are useful tools for CPU intensive heavy tasks which otherwise would halt your sychronous workflow. However since JS is an evented language, for all async tasks spawning promises one after the other synchronously (such as `Promise.all()`) and wait for their resolutions in a parallel manner is very efficient. – Redu Apr 18 '18 at 10:36