Promise All retry

Question

I know that promise.all() fails when even 1 of the promise is failed. I only want to try for failed promises and don't want to run promise.all() again.

Any recommendations on how I can achieve this in minimal way?

one simple trick is to wrap each promise(returns success or fail) with another promise(which always returns success). then count the failures and successes with promise.all(). — DragonKnight, Sep 03 '18 at 11:17
Yep, boxing the results inside a `{ success: bool, value: any }` is good, but you also have to give a `retry` function for failed promises. — adz5A, Sep 03 '18 at 11:50

adz5A · Accepted Answer · 2018-09-04T07:03:29.757

Promises are eager construct and model a value obtained asynchronously, a Promise is produced using some kind of producer, like fetch for instance.

If you retain a reference to this producer then you can replay the nmechanism that produced the Promise in the first place.

// producer function
function getData (arg) {
  const result = new Promise();
  return result.then(value => {
    return { ok:true, value };
  }, error => {
    return {
      ok: false,
      value: error,
      // retry is a function which calls the producer with the same arguments
      retry: () => getData(arg)
    };
  })

}

Then if you have something like:

const data = [];

// Promise<{ok: boolean, value: any, retry?: function}>
// No promises will fail in this array
const asyncResults = data.map(getResults); 


Promise.all(asyncResults)
  .then((results) => {
    const successes = results.filter(res => res.ok);
    const retrys = results.filter(res => !res.ok).map(res => res.retry()); // retry all failed promises
  })

Memory leaks, stack overflow: because I retain a reference to original arguments in order to retry and the algorithm is recursive there could be a memory leak. However the algorithm cannot "stack overflow":

getData calls do not get "deeper" over time (see retry definition)
the asyncrhonicity of the algorithm prevent this behaviour if a promise was never resolved
old data is properly discarded when accessing the results as const resultData = results.filter(res => res.ok).map(res => res.value);

However the algorithm could take a long time to settle if a promise keep on not getting resolved and prevent access to the rest of the values.

In an alternative I suggest you take a look at another async primitive, not yet part of the language (maybe some day) : Observables which are designed for this kind of tasks: lazy, retry-able async operations.

I did not mention it in the answer but the whole structure of this is recursive. `retrys` is an array of promise which needs to be merged with the successes values. — adz5A, Sep 03 '18 at 12:23
as it's recursive, is there any chance to be out of stack memory? — Talha Irfan, Sep 04 '18 at 05:32
I updated for you comment. My answer lacks the end of the algorithm, left as an exercise (actually multiple ways to handle it depending on your use case and data), do you have any more questions ? — adz5A, Sep 04 '18 at 11:25
no, it doesn't solves my scenario problem. But, was good thing to learn — Talha Irfan, Sep 04 '18 at 12:17

num8er · Answer 2 · 2022-02-03T23:04:31.177

You may use async package and wrap all promise calls with closure with done argument.

Then simply resolve results.

const async = require('async');

const promiseAll = promises => {
  return new Promise((resolve) => {
    
   // preparing array of functions which has done method as callback
   const parallelCalls = promises.map(promise => {
      return done => {
        promise
          .then(result => done(null, result)
          .catch(error => {
            console.error(error.message);
            done();
          });
      }
    });
    
    // calling array of functions in parallel
    async.parallel(
      parallelCalls, 
      (_, results) => resolve(results.filter(Boolean))
    );

  });
};


router.get('/something', async (req, res) => {
  ...
  const results = await promiseAll(promises);
  ...
});

Or we can simply do Promise.all without using async package:

router.get('/something', async (req, res) => {
  ...
  const results = (
    await Promise.all(
      promises.map(promise => {
        return new Promise(resolve => {
          promise.then(resolve).catch(e => resolve());
        });
      });
    )
  ).filter(Boolean);
  ...
});

Promise All retry

2 Answers2