0

I'm using node.js, have a graph of dependent REST calls and am trying to dispatch them in parallel. It's part of a testing/load testing script.

My graph, has "connected components", and each component is directed and acyclic. I toposort each component, so I end up with a graph that looks like this

Component1 = [Call1, Call2...., Callm] (Call2 possibly dependent on call1 etc)
Component2 = [Call1, Call2... Calln]
...
Componentp

The number of components, and calls in each component m, n and p are dynamic

I want to round robin over the components, and each of it's calls, dispatching up to "x" calls concurrently.

Whilst I understand a little about Promises, async/await and Node's event loop I'm NOT an expert.

PSEUDO CODE ONLY

maxParallel = x
runningCallCount = 0
while(components.some(calls => calls.some(call => noResponseYet(call)) {
    if (runningCallCount < maxParallel) {
        runningCallCount++
        var result = await axios(call)
        runningCallCount--
    }
}

This doesn't work - I never dispatch the calls. Remove the await and i fall through to the runningCallCount-- straight away.

Other approaches I've tried and comments

  • Wrapping every call in an async function, and using Promise.All on a chunk of x number at a time - a chunking style of approach. This may work, but It doesn't acheive the result of allways trying to have x parallel calls going
    • Used RxJs - tried merge on all components with a max number of parallelism - but this parallelises the components, not the calls within the components, and i couldn't work out how to make it work the way i wanted based on the poor doco. I'd used the .NET version before so this was a bit disappointing.
    • I haven't yet tried recursion

Can anyone chime in with an idea as to how to do this ? How does await work in node ? I've seen it explained like generator functions and yield statements (https://medium.com/siliconwat/how-javascript-async-await-works-3cab4b7d21da) Can anyone add detail - how is the event loop checked when code strikes an await call ? Again I'm guessing either the entire stack unrolls, or a call to run the event loop is somehow inserted by the await call.

I'm not interested in using a load testing package, or other load testing tools - I just want to understand the best way to do this, but also understand what's going on in node and await.

I'll update this if i understand this or find a solution, but

Help appreciated.

Disgruntled Goat
  • 107
  • 2
  • 7

1 Answers1

0

I would think something like this would work to achieve always having n parallel calls going. 

const delay = time => new Promise(r=>setTimeout(r,time));

let maxJobs = 4;
let jobQueue = [
  {time:1000},{time:3000},{time:1000},{time:2000},
  {time:1000},{time:1000},{time:2000},{time:1000},
  {time:1000},{time:5000},{time:1000},{time:1000},
  {time:1000},{time:7000},{time:1000},{time:1000}
];
jobQueue.forEach((e,i)=>e.id=i);

const jobProcessor = async function(){
  while(jobQueue.length>0){
    let job = jobQueue.pop();
    console.log('Starting id',job.id);
    await delay(job.time);
    console.log('Finished id',job.id);
  }
  return;
};

(async ()=>{
  console.log("Starting",new Date());
  await Promise.all([...Array(maxJobs).keys()].map(e=>jobProcessor()))
  console.log("Finished",new Date());
})();
Cody Ｇ
  • 8,368
  • 2
  • 35
  • 50
  • Thanks @Cody G. - that works a treat, and is neatly done. I pursued a similar idea, but it didn't work. I suspect what was different was that mine wasn't inside a map call. As per https://stackoverflow.com/questions/37576685/using-async-await-with-a-foreach-loop there is some state machine magic going on that i still don't grok. Any clues how the event loop is being checked inside the Promise.all call ? – Disgruntled Goat Aug 30 '18 at 00:38
  • So the `Promise.all` I'm using here is really just waiting for the end of all the job processors. It actually has nothing to do with processing the jobs, per say. Put another way, the event loop is just checking whether or not all the `jobProcessor()` promises (4 of them) have been completely resolved. Due to the "single threaded" nature of javascript we can use a single array between all of the `jobProcessors` which allows them to keep popping a new job off the shared job stack. – Cody Ｇ Aug 30 '18 at 12:56
  • You don't need to answer this @Cody G. your solution has already helped me understand whats going on better - I think I need to up my understanding more to clarify my question around how this await **works**. But for anyone else reading this, with a similar question - "Any clues how the event loop is being checked inside the Promise.all" - this is the best links i've found so far - https://itnext.io/javascript-promises-with-node-js-e8ca827e0ea3. Coming from the lower level - http://docs.libuv.org/en/v1.x/async.html, Does the javascript stack with the await in it "run to completion" ? TBC – Disgruntled Goat Aug 31 '18 at 00:35
  • It's good that you take an interest in the lower level workings of a library. I could probably use a little more of that but I don't often research performance because it runs "fast enough" --- until it doesn't! – Cody Ｇ Aug 31 '18 at 12:56