9

I saw this example implementation of Promise.all - which runs all promises in parallel - Implementing Promise.all

Note that the functionality I am looking for is akin to Bluebird's Promise.mapSeries http://bluebirdjs.com/docs/api/mapseries.html

I am making an attempt at creating Promise.series, I have this which seems to work as intended (it actually is totally wrong, don't use it, see answers):

Promise.series = function series(promises){

    return new Promise(function(resolve,reject){

    const ret = Promise.resolve(null);
    const results = [];

    promises.forEach(function(p,i){
         ret.then(function(){
            return p.then(function(val){
               results[i] = val;
            });
         });
    });

    ret.then(function(){
         resolve(results);
    },
     function(e){
        reject(e);
     });

    });

}


Promise.series([
    new Promise(function(resolve){
            resolve('a');
    }),
    new Promise(function(resolve){
            resolve('b');
    })
    ]).then(function(val){
        console.log(val);
    }).catch(function(e){
        console.error(e.stack);
    });

However, one potential problem with this implementation is that if I reject a promise, it doesn't seem to catch it:

 Promise.series([
    new Promise(function(resolve, reject){
            reject('a');   // << we reject here
    }),
    new Promise(function(resolve){
            resolve('b');
    })
    ]).then(function(val){
        console.log(val);
    }).catch(function(e){
        console.error(e.stack);
    });

does anyone know why the error doesn't get caught and if there is a way to fix this with Promises?

According to a comment, I made this change:

Promise.series = function series(promises){

    return new Promise(function(resolve,reject){

    const ret = Promise.resolve(null);
    const results = [];

    promises.forEach(function(p,i){
         ret.then(function(){
            return p.then(function(val){
               results[i] = val;
            },
            function(r){
                console.log('rejected');
                reject(r);   // << we handle rejected promises here
            });
         });
    });

    ret.then(function(){
         resolve(results);
    },
     function(e){
        reject(e);
     });

    });

}

but this still doesn't work as expected...

Community
  • 1
  • 1
Alexander Mills
  • 90,741
  • 139
  • 482
  • 817
  • 2
    You're not catching the inner promise. Try to use `reduce()` instead of `forEach`, chaining them by `then` – salezica Jun 01 '16 at 21:21
  • I don't see how your `Promise.series` is different from `Promise.all`. It seems to do exactly the same – Eugene Jun 01 '16 at 21:22
  • it's different :) it runs the promises array in series instead of in parallel, as you might expect – Alexander Mills Jun 01 '16 at 21:23
  • @Zhegan I think the difference is running in series vs parallel. – Mulan Jun 01 '16 at 21:24
  • 5
    But it doesn't "run" promises. `Promise.series` gets already created promises that are running right after they are created. – Eugene Jun 01 '16 at 21:30
  • @Zhegan promises aren't run until then() is called on them :) – Alexander Mills Jun 01 '16 at 21:31
  • 3
    No, when you create promise via `new Promise(executor)` the `executor` function is invoked upon creation (moreover synchronously). Inside `Promise.series` you can only wait for promises fulfillment/rejection. – Eugene Jun 01 '16 at 21:38
  • 4
    @AlexMills No, `then()` doesn't "start" a promise. Both `Promise.all([$.get("foo.html"), $.get("bar.html")])` and `Promise.series([$.get("foo.html"), $.get("bar.html")])` would immediately start 2 AJAX requests in parallel and then await their results. If you want the next request to start after the previous one completed, you need to give `Promise.series` an array of "promise factories": functions that *create* (and thus "start") a promise. – Mattias Buelens Jun 01 '16 at 21:40
  • 4
    I would pass array of functions (and call it tasks) to the `Promise.series` and execute them in series. Passing array of promises just does not make sense for me - it works almost the same as `Promise.all` – Eugene Jun 01 '16 at 21:41
  • Also please don't add arbitrary functions like `series` onto global object. You'd be much better off making a module than exports the `series` function and then imports it where it is needed. – loganfsmyth Jun 01 '16 at 22:03
  • Yes, I wouldn't do that, it's just an example :) – Alexander Mills Jun 01 '16 at 22:04
  • 1
    Perfect, never hurts to make sure :) – loganfsmyth Jun 01 '16 at 22:09

4 Answers4

4

The promise returned by then in the forEach loop does not handle potential errors.

As pointed out in a comment by @slezica, try to use reduce rather than forEach, this chains all promises together.

Promise.series = function series(promises) {
    const ret = Promise.resolve(null);
    const results = [];

    return promises.reduce(function(result, promise, index) {
         return result.then(function() {
            return promise.then(function(val) {
               results[index] = val;
            });
         });
    }, ret).then(function() {
        return results;
    });
}

Keep in mind that the promises are already "running" at that point though. If you truly want to run your promises in series, you should adjust your function and pass in an array of functions that return promises. Something like this:

Promise.series = function series(providers) {
    const ret = Promise.resolve(null);
    const results = [];

    return providers.reduce(function(result, provider, index) {
         return result.then(function() {
            return provider().then(function(val) {
               results[index] = val;
            });
         });
    }, ret).then(function() {
        return results;
    });
}
Alexander Mills
  • 90,741
  • 139
  • 482
  • 817
forrert
  • 4,109
  • 1
  • 26
  • 38
  • Can you make it a little bit clearer why reduce is better than forEach? – Alexander Mills Jun 01 '16 at 21:33
  • When you call `promise.then` a new promise is returned. In your original version this new promise was "lost" in the `forEach` loop. When using `reduce` you "chain" the initial promise through the array. – forrert Jun 01 '16 at 21:39
  • ahhh, I think I see what you mean – Alexander Mills Jun 01 '16 at 21:40
  • 1
    Keep in mind @zhegan's comments though. The promises are already "running" when you pass them in your function in the array. If you really want a series of promises to be run, you should pass in an array of functions that create your promises once the previous promise is resolved. – forrert Jun 01 '16 at 21:41
  • You are right, Promise.all takes an array of promises. But as you correctly noticed, in that case the promises are run in parallel. That's because they are already running in that case. As others have pointed out, without passing in an array of functions, you'll end up running the promises in parallel again... – forrert Jun 01 '16 at 21:50
  • 4
    Calling `.then()` is **not** supposed to affect the execution of a Promise. Any implementation that does so is out-of-spec; this behaviour should not be relied on. – Jeremy Jun 01 '16 at 21:51
  • yes you are both correct, my mistake, the promises are already running! – Alexander Mills Jun 01 '16 at 21:54
  • 1
    This runs all the promises in parallel because the chain is always started with `Promise.resolve(null).then(promise[0]).then...` Because the first promise is resolve, subsequent `.then` calls do not *wait* for any promises down the line. The error will be evident if you try running my code but replacing the `Promise.series` function with yours. – Mulan Jun 01 '16 at 22:00
  • @naomik in my brief testing this is not the case, I will post an answer and you can test it it out – Alexander Mills Jun 01 '16 at 23:58
4

This is a common misunderstanding of how promises work. People want there to be a sequential equivalent to the parallel Promise.all.

But promises don't "run" code, they're mere return values one attaches completion callbacks to.

An array of promises, which is what Promise.all takes, is an array of return values. There's no way to "run" them in sequence, because there's no way to "run" return values.

Promise.all just gives you one promise representing many.

To run things in sequence, start with an array of things to run, i.e. functions:

let p = funcs.reduce((p, func) => p.then(() => func()), Promise.resolve());

or an array of values to run a function over:

let p = values.reduce((p, val) => p.then(() => loadValue(val)), Promise.resolve());

Read up on reduce here.

Update: Why Promises don't "run" code.

Most people intuitively understand that callbacks don't run in parallel.

(Workers aside,) JavaScript is inherently event-driven and single-threaded, and never runs in parallel. Only browser functions, e.g. fetch(url) can truly do work in parallel, so an "asynchronous operation" is a euphemism for a synchronous function call that returns immediately, but is given a callback (e.g. where resolve would be called) that will be called later.

Promises don't change this reality. They hold no inherent asynchronous powers (*), beyond what can be done with callbacks. At their most basic, they're a (very) neat trick to reverse the order in which you need to specify callbacks.

*) Technically speaking, promises do have something over callbacks, which is a micro-task queue in most implementations, which just means promises can schedule things at the tail of the current crank of the JavaScript event-loop. But that's still not vastly different, and a detail.

Community
  • 1
  • 1
jib
  • 40,579
  • 17
  • 100
  • 158
  • you should look at the other answers and adjust yours accordingly – Alexander Mills Jun 02 '16 at 16:49
  • if promises "don't run code" then can you explain what the function with the signature function(resolve,reject){} does that's passed to the Promise constructor? :) – Alexander Mills Jun 02 '16 at 17:21
  • @AlexMills that's the *Promise constructor executor function*. It runs and completes before you even get the promise. – jib Jun 02 '16 at 17:31
  • to answer my own question, that function is actually what "runs code" and resolve and reject serve as callbacks. So by "running promises in series or in parallel", the code that is defined in this function is what in effect will run in series or in parallel. – Alexander Mills Jun 02 '16 at 17:31
  • @jip, no, not necessarily at all, if you make some async call that takes a second or two, it's not necessarily resolved/rejected yet, furthermore, if you wrap the promises in provider functions (as you have suggested) they won't have begun executing at all. – Alexander Mills Jun 02 '16 at 17:33
  • you have completely bemused me, because you clearly have a good understanding of this, but seem to be completely wrong about promises "not running code", what does that even mean? – Alexander Mills Jun 02 '16 at 17:34
  • promises are primarily designed for async operations, and that async operation is "hidden" in the Promise constructor executor function, but it's still there, and that's where you implement the async behavior, and those resolve/reject callbacks may not fire anytime soon. – Alexander Mills Jun 02 '16 at 17:35
  • also, note that Bluebird has a .mapSeries utility, http://bluebirdjs.com/docs/api/mapseries.html, explain that :) – Alexander Mills Jun 02 '16 at 17:45
  • @AlexMills Workers aside, JavaScript is inherently event-driven and single-threaded, and never runs in parallel. Only browser functions, e.g. `fetch(url)` can truly do work in parallel, so an "asynchronous operation" is a euphemism for a synchronous function call that returns immediately, that's given a callback (where you typically call `resolve`) that will be called *later*. Therefore, the executor function is just another JS function that runs *synchronously*. Looking at the Promise constructor is a bad place to start to learn promises, because it only exists to wrap old APIs. – jib Jun 02 '16 at 17:56
  • that last statement is 100% false! even in a promise-only world with no old APIs, we still need that executor function to handle any asynchronous operations, the purpose of promises in the first place. It's not turtles (promises) all the way down, at some point you will reach callbacks. I take it that you have never written a library that exposes promises instead of callbacks. – Alexander Mills Jun 02 '16 at 18:01
  • @AlexMills I think you are mistaken. FWIW I work on the Firefox browser, and I introduced promises to the WebRTC APIs. – jib Jun 02 '16 at 18:11
  • Well, until V8 and Node core expose promises from internal APIs, will be using the Promise constructor executor function, which "runs code", and most likely makes an async call, which allows Promises to interleave. – Alexander Mills Jun 02 '16 at 18:16
  • You may be thinking from a front-end perspective and I am thinking backend. But my argument still applies. – Alexander Mills Jun 02 '16 at 18:35
  • @AlexMills I think you're confusing a *promise* (an object) with the promise *constructor* (a function). The latter does execute a passed-in function immediately, where you can call an old callback API e.g. `setTimeout(resolve, 1000)`, but then it returns immediately, whereas `resolve` gets called a second later. Event-driven, and no special magic. – jib Jun 02 '16 at 18:39
3

EDIT 2

According to your edit, you're looking for Promise.mapSeries as provided by bluebird. You've given us a bit of a moving target, so this edit changes direction from my previous answer because the mapSeries function works very differently than just executing a collection of Promises in serial order.

// mock bluebird's mapSeries function
// (Promise [a]) -> (a -> b) -> (Promise [b])
Promise.prototype.mapSeries = function mapSeries(f) {
  return this.then(reducek (ys=> x=> k=> {
    let value = f(x);
    let next = x=> k([...ys, x]);
    return value instanceof Promise ? value.then(next) : next(value);
  }) ([]));
};

Just to get a top-level idea of how this would be used

// given: (Promise [a]) and (a -> b)
// return: (Promise [b])
somePromiseOfArray.mapSeries(x=> doSomething(x)); //=> somePromiseOfMappedArray

This relies on a small reducek helper which operates like a normal reduce except that the callback receives an additional continuation argument. The primary advantage here is that our reducing procedure has the option of being asynchronous now. The computation will only proceed when the continuation is applied. This is defined as a separately because it's a useful procedure all on its own; having this logic inside of mapSeries would make it overly complicated.

// reduce continuation helper
// (a -> b -> (a -> a)) -> a-> [b] -> a
const reducek = f=> y=> ([x, ...xs])=> {
  if (x === undefined)
    return y;
  else
    return f (y) (x) (y => reducek (f) (y) (xs));
};

So you can get a basic understanding of how this helper works

// normal reduce
[1,2,3,4].reduce((x,y)=> x+y, 0); //=> 10

// reducek
reducek (x=> y=> next=> next(x+y)) (0) ([1,2,3,4]); //=> 10

Next we have two actions that we'll use in our demos. One that is completely synchronous and one that returns a Promise. This demonstrates that mapSeries can also work on iterated values that are Promises themselves. This is the behaviour defined by bluebird.

// synchronous power
// Number -> Number -> Number
var power = x=> y=> Math.pow(y,x);

// asynchronous power
// Number -> Number -> (Promise Number)
var powerp = x=> y=>
  new Promise((resolve, reject)=>
    setTimeout(() => {
      console.log("computing %d^%d...", y, x);
      if (x < 10)
        resolve(power(x)(y));
      else
        reject(Error("%d is just too big, sorry!", x));
    }, 1000));

Lastly, a small helper used to facilitate logging in the demos

// log promise helper
const logp = p=>
  p.then(
    x=> console.log("Done:", x),
    err=> console.log("Error:", err.message)
  );

Demo time! Here I'm going to dogfood my own implementation of mapSeries to run each demo in sequential order!.

Because mapSeries excepts to be called on a Promise, I kick off each demo with Promise.resolve(someArrayOfValues)

// demos, map each demo to the log
Promise.resolve([

  // fully synchronous actions map/resolve immediately
  ()=> Promise.resolve([power(1), power(2), power(3)]).mapSeries(pow=> pow(2)),

  // asynchronous items will wait for resolve until mapping the next item
  ()=> Promise.resolve([powerp(1), powerp(2), powerp(3)]).mapSeries(pow=> pow(2)),

  // errors bubble up nicely
  ()=> Promise.resolve([powerp(8), powerp(9), powerp(10)]).mapSeries(pow=> pow(2))
])
.mapSeries(demo=> logp(demo()));

Go ahead, run the demo now

// reduce continuation helper
// (a -> b -> (a -> a)) -> a-> [b] -> a
const reducek = f=> y=> ([x, ...xs])=> {
  if (x === undefined)
    return y;
  else
    return f (y) (x) (y => reducek (f) (y) (xs));
};

// mock bluebird's mapSeries function
// (Promise [a]) -> (a -> b) -> (Promise [b])
Promise.prototype.mapSeries = function mapSeries(f) {
  return this.then(reducek (ys=> x=> k=>
    (x=> next=>
      x instanceof Promise ? x.then(next) : next(x)
    ) (f(x)) (x=> k([...ys, x]))
  ) ([]));
};

// synchronous power
// Number -> Number -> Number
var power = x=> y=> Math.pow(y,x);

// asynchronous power
// Number -> Number -> (Promise Number)
var powerp = x=> y=>
  new Promise((resolve, reject)=>
    setTimeout(() => {
      console.log("computing %d^%d...", y, x);
      if (x < 10)
        resolve(power(x)(y));
      else
        reject(Error("%d is just too big, sorry!", x));
    }, 1000));


// log promise helper
const logp = p=>
  p.then(
    x=> console.log("Done:", x),
    err=> console.log("Error:", err.message)
  );

// demos, map each demo to the log
Promise.resolve([

  // fully synchronous actions map/resolve immediately
  ()=> Promise.resolve([power(1), power(2), power(3)]).mapSeries(pow=> pow(2)),

  // asynchronous items will wait for resolve until mapping the next item
  ()=> Promise.resolve([powerp(1), powerp(2), powerp(3)]).mapSeries(pow=> pow(2)),

  // errors bubble up nicely
  ()=> Promise.resolve([powerp(8), powerp(9), powerp(10)]).mapSeries(pow=> pow(2))
])
.mapSeries(f=> logp(f()));

EDIT

I'm reapproaching this problem as a series of promises should be considered like a chain or composition of promises. Each resolve promise will feed it's value to the next promise.

Per @Zhegan's remarks, it makes more sense for the series function to take an array of promise creators, otherwise there's no way to guarantee the promises would run in serial. If you pass an array of Promises, each promise will immediately run its executor and start doing work. Thus, there's no way that the work of Promise 2 could depend on the completed work of Promise 1.

Per @Bergi's remarks, my previous answer was a little weird. I think this update makes things a little more consistent.

Promise series without error

// ([(a-> (Promise b)), (b-> (Promise c)]), ...]) -> a -> (Promise c)
Promise.series = function series(tasks) {
  return x=>
    tasks.reduce((a,b)=> a.then(b), Promise.resolve(x));
};

// a -> [a] -> (Promise [a])
var concatp = x=> xs=>
  new Promise((resolve, reject)=>
    setTimeout(() => {
      console.log(xs, x);
      if (xs.length < 3)
        resolve(xs.concat([x]));
      else
        reject(Error('too many items'));
    }, 250));

var done = (x)=> console.log('done:', x);
var err = (e)=> console.log('error:', e.message);

Promise.series([concatp(3), concatp(6), concatp(9)]) ([]) .then(done, err);
// [] 3
// [ 3 ] 6
// [ 3, 6 ] 9
// done: [ 3, 6, 9 ]

Promise series with an error

// ([(a-> (Promise b)), (b-> (Promise c)]), ...]) -> a -> (Promise c)
Promise.series = function series(tasks) {
  return x=>
    tasks.reduce((a,b)=> a.then(b), Promise.resolve(x));
};

// a -> [a] -> (Promise [a])
var concatp = x=> xs=>
  new Promise((resolve, reject)=>
    setTimeout(() => {
      console.log(xs, x);
      if (xs.length < 3)
        resolve(xs.concat([x]));
      else
        reject(Error('too many items'));
    }, 250));

var done = (x)=> console.log('done:', x);
var err = (e)=> console.log('error:', e.message);

Promise.series([concatp(3), concatp(6), concatp(9), concatp(12)]) ([]) .then(done, err);
// [] 3
// [ 3 ] 6
// [ 3, 6 ] 9
// [ 3, 6, 9 ] 12
// error: too many items
Mulan
  • 129,518
  • 31
  • 228
  • 259
  • 1
    Your implementation only produces the result from the last promise, rather than an array of all results. It also passes the result from the previous promise to the next promise provider, which is not what it should do. Finally, it's quite confusing to have the first array element be a promise and the others be promise providers... – Mattias Buelens Jun 01 '16 at 22:09
  • I do not believe that the promises should some how *collect* the data in an array. Rather that data should move through the promises in sequence. If you wish to collect data, that should be reflected in the task themselves. `Promise.series` should not have an influence there. – Mulan Jun 01 '16 at 22:14
  • It's like `Array.prototype.reduce`; it doesn't force a `concat` on you but you could easily write a reducer that *does* use `concat` if that's the behaviour you wanted. Anyway, it doesn't matter. `Promise.series` isn't a real function, so we can make it behave however you want. This is how mine behaves. – Mulan Jun 01 '16 at 22:15
  • "*Promise.series expects at least 1 promise*" is very weird. Also, it currenctly appears to actually expect one promise and zero or more *functions* in that array. Why not just use the `start` argument of `reduce`? – Bergi Jun 01 '16 at 23:23
  • I like this solution, but from what I have gathered so far, you can use Promise.resolve() to create a Promise that does not resolve in the same tick, so this works, and you can use that to seed the reduce thing – Alexander Mills Jun 01 '16 at 23:57
  • Oh I'm so glad about your third revision :-) – Bergi Jun 02 '16 at 00:00
  • 1
    @Bergi thanks for encouraging me to rethink the problem. I think this edit might be a little better. – Mulan Jun 02 '16 at 00:00
  • frankly, even for an experienced JS dev, it's hard to parse the arrow functions in your answer. if you want this to be intelligible to anyone wishing to learn the concept behind it, you may wish to de-obfuscate, otherwise good work thanks – Alexander Mills Jun 02 '16 at 17:38
  • @AlexMills if there's some ES6 that's hard for you to read/understand, I recommend the [babel.js repl](http://babeljs.io/repl/) tool. Maybe one of the nicest things about Babel is that the code output maintains excellent readability for most things. – Mulan Jun 02 '16 at 22:28
  • @AlexMills since you've now stated that you're looking to copy bluebird's `mapSeries` functionality, I've updated my answer yet again. – Mulan Jun 03 '16 at 20:17
  • @AlexMills Arrows are standardized since ES6. As long as you don't depend on a dynamic `this` or `super`, there is no reason not to use them. –  Jun 03 '16 at 21:45
2

@forrert's answer is pretty much spot on

Array.prototype.reduce is a bit confusing, so here is a version without reduce. Note that in order to actually run promises in series we must wrap each promise in a provider function and only invoke the provider function inside the Promise.series function. Otherwise, if the promises are not wrapped in functions, the promises will all start running immediately and we cannot control the order in which they execute.

Promise.series = function series(providers) {

    const results = [];
    const ret = Promise.resolve(null);

    providers.forEach(function(p, i){
         ret = ret.then(function(){
            return p().then(function(val){
                  results[i] = val;
            });
         });
    });

    return ret.then(function(){
         return results;
    });

}

the equivalent functionality using reduce:

Promise.series = function series(providers) {
    const ret = Promise.resolve(null);
    const results = [];

    return providers.reduce(function(result, provider, index) {
         return result.then(function() {
            return provider().then(function(val) {
               results[index] = val;
            });
         });
    }, ret).then(function() {
        return results;
    });
}

you can test both functions using this:

Promise.series([

    function(){
      return new Promise(function(resolve, reject){
          setTimeout(function(){
              console.log('a is about to be resolved.')
              resolve('a');
          },3000);  
       })   
    },
    function(){
        return new Promise(function(resolve, reject){
            setTimeout(function(){
                  console.log('b is about to be resolved.')
                  resolve('b');
            },1000);
        })
    }   

    ]).then(function(results){
        console.log('results:',results);
    }).catch(function(e){
        console.error('Rejection reason:', e.stack || e);
    });

note that it's not a good idea to attach functions, or otherwise alter, native global variables, like we just did above. However, also note that the native library authors also left us with native libraries that are wanting in functionality :)

Alexander Mills
  • 90,741
  • 139
  • 482
  • 817
  • But there's no way for the work of Promise *N* to depend on the completed work of a previous Promise. I know this may not be a requirement, but it vastly limits the types of functions you can have in your sequence. – Mulan Jun 02 '16 at 00:08
  • that may be true, but you are thinking of async.waterfall type functionality...but what I am looking for is async.mapSeries functionality....we want a mapping action, which happens in series, not a chaining action that happens in series. basically, like the title of the question says, we want Promise.all but in series! – Alexander Mills Jun 02 '16 at 00:10
  • I am not sure how it limits the types of functions you can have? You can have any function you want and it will capture the then value of the returned promise – Alexander Mills Jun 02 '16 at 00:11
  • But `map` *is* a `reduce`! `var map = (f,xs)=> xs.reduce((ys,x)=> ys.concat([f(x)]), []);`. Then `map(x=>x*x, [1,2,3]); //=> [1,4,9]` – Mulan Jun 02 '16 at 00:13
  • I am not following, I am not sure I agree that mapping is a reduce action nor what your point is? – Alexander Mills Jun 02 '16 at 00:15