What is a good pattern for waiting until x number of promises are completed to spawn a new promise?

Question

I have been searching and have found some variations of this but have not found anything that meets my needs exactly. Below is the pseudocode that I am trying to figure out:

var itemsUploading = 0;
const maxUploads = 10;


function uploadAll(items){
    var promises = [];
    items.forEach(function(item){

        /* Here is where I want to wait for itemsCurrentlyUploading < maxUploads */

        itemsUploading++;            
        promises.push(upload(item));            
    }
    return Promise.all(promises);

}

function upload(item){
    return new Promise(function(resolve, reject){

        /*Item upload logic here */

    })
    .then(function(response){
        itemsUploading--;
    });
}

I do understand that you cannot block synchronous code to wait for asynchronous and I have tried several variations. I think setTimeout is something that I can use to do this but I can't quite understand the logic necessary. Any and all help would be greatly appreciated and if there is anything else I can edit this with to help someone understand the problem better just let me know, thank you!

You need to read up on chaining Promises and Deferred calls, keyword here is `.then()` — rorschach, Nov 22 '16 at 17:35
Have a look at [this](http://stackoverflow.com/a/39197252/1048572) or [this](http://stackoverflow.com/a/38778887/1048572) — Bergi, Nov 22 '16 at 17:44

score 1 · Accepted Answer · answered Nov 22 '16 at 18:05

1

I think you may find something like Promise.map from Bluebird library useful. concurrency parameter allows you to set maximum number of "pending" promises.

Simple solution, but I guess not as battle-tested as Bluebird, is available as standalone NPM package.

answered Nov 22 '16 at 18:05

Eryk Napierała

516
4
7

This is perfect, exactly what I was looking for. – goodOldFashioned Nov 22 '16 at 19:33

Jared Smith · Answer 2 · 2016-11-22T17:49:14.473

Since JavaScript is a little sparse on the data structures, we'll have to make do.

let pending = new Set;
let maxPendingItems = 5; // could be whatever number

let wrapPromiseFn = f => (...args) => {
  let p = f(...args);
  pending.add(p);
  p
    .then(() => pending.remove(p))
    .catch(e => {
      console.warn(e);
      pending.remove(p);
    });

  return p;
};

let performAsyncIO = wrapPromiseFn(someAJAXFn);

Now you can have a loop that monitors the size of the set to see if it has cleared out enough to add new items. Just make sure that you decorate your Promise-returning functions with wrapPromiseFn like above.

let flag = true;
while (flag) {
  if (pending.size <= maxPendingItems) {
    // promise will automatically queue until resolved
    performAsyncIO().then(responseHandler);
  }
}

What is a good pattern for waiting until x number of promises are completed to spawn a new promise?

2 Answers2