0

I want to run an async function N times but for performance reasons have M calls running simultaneously. So at the start I would fire off two function calls and then when the first of those finishes I should start the 3rd and, when the next finishes start the 4th. When the last has started I then need to wait for the final two to finish.

I should never have more than 2 running at any time.

I've been looking at how to do this with Promise.any() but not quite been able to work it out.

kelsny
  • 23,009
  • 3
  • 19
  • 48
Howard May
  • 6,639
  • 9
  • 35
  • 47
  • So like... pooling the promise results in a way? – kelsny Oct 28 '22 at 18:52
  • Like [this](https://tsplay.dev/Nl2dOW) maybe? There are libraries that do this (I think "p-limit" is one of them) but suggestions for libraries are usually out of scope for Stack Overflow. Could you provide a [mre] of what you mean, though? Like, write some code that has some toy async functions you want to throttle so that we can measure what success or failure looks like – jcalz Oct 28 '22 at 19:24
  • Check out `mapConcurrent()` [here](https://stackoverflow.com/questions/46654265/promise-all-consumes-all-my-ram/46654592#46654592). This iterates an array N at a time calling an asynchronous function on each item, but the concept can certainly be adapted to not involve an array (or you can make a dummy array to feed it as is). – jfriend00 Oct 28 '22 at 21:05
  • Or perhaps `runN()` [here](https://stackoverflow.com/questions/48842555/loop-through-an-api-get-request-with-variable-url/48844820#48844820) might be well suited too. – jfriend00 Oct 28 '22 at 21:06
  • Is this what you need? [Async task manager with maximum number of concurrent "running" tasks](https://stackoverflow.com/questions/54901078/async-task-manager-with-maximum-number-of-concurrent-running-tasks) – trincot Oct 29 '22 at 07:49

1 Answers1

0

I would use the queue function in the async library.

You would create a queue:

  • That calls your function for each item in the queue
  • That has a concurrency limit of M
  • And place N objects in the queue

The code would look something like:

const q = async.queue(function(data, callback) {
    myFunc()
    callback()
}, M)

q.push("item1")
q.push("item2")
q.push("item3")

etc

matt helliwell
  • 2,574
  • 16
  • 24