resolving Promises sequentially

Question

EDIT: This is for an Electron project, with a local server spun up on the user's system. So, any concerns about what happens if multiple users attempt simultaneous access can be ignored!

My client code is generating an array of JavaScript objects, which I'm calling packets. There can be potentially infinite packets, although between 1 and 10 is the most common use-case.

I need to make API calls to a backend route, one per packet, passing a packet as an argument to each call.

However, the backend does some heavy computational work on each packet, to the point where attempting to crunch more than 2-3 packets at once crashes the server.

Is it possible to resolve Promises synchronously, such that the second Promise only fires when the first resolves, the third firing after the second, and so on?

It's my understanding that Promise.all() makes all calls simultaneously, which doesn't help me here.

(I also know that this is an anti-pattern, but for my specific set of requirements, I'm not sure what else to do)

I know this is purely abstract, but any thoughts would be appreciated!!!

Yeah 'sequentially' would've been better semantics. I just meant that Promise 1 needs to finish before Promise 2 starts, Promise 2 needs to finish before Promise 3 starts and so on, up until Promise N. — Adam Templeton, Oct 31 '17 at 23:28
`My client code is generating an array of JavaScript objects` - let me ask this. Are you creating the array, and once the array is generated, then you make the API calls? Or is it more of a situation where these objects are created on the fly and you want to ensure you only call the API one at a time - i.e. the "queue" answer below would be ideal — Jaromanda X, Oct 31 '17 at 23:39
Sounds like you're just describing [chaining](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Guide/Using_promises#Chaining). Use [composition](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Guide/Using_promises#Composition). — jib, Nov 01 '17 at 01:37
One option is to think of this as a [Promise-throttling problem](https://stackoverflow.com/q/38385419/1426891) where `n = 1`, though it sounds like you'd be comfortable with `n = 2` or `n = 3` as well. — Jeff Bowman, Nov 01 '17 at 06:16

Rafael · Accepted Answer · 2017-10-31T23:41:21.090

3

Get weird with Promises

An async queue, a spin-off one of my previous answers; I've added random completion time to simulate a real environment:

class Queue {
    constructor() {
        this.queue = [];
    }

    enqueue(obj) {
        return this.queue.push(obj);
    }

    dequeue() {
        return this.queue.shift();
    }

    hasWork() {
        return (this.queue.length > 0);
    }
}

class AsyncQueue extends Queue {
    constructor(job) {
        super();
        this.job = job;
    }

    process(cb) {
        return this.job(this.dequeue()).then(data => {
            cb(data);
            if (this.hasWork())
                return this.process(cb);
        });
    }
}

//MUST RETURN Promise
function work() {
    var duration = chooseDelay();
    console.log('START JOB, I.E., MAKE REQUEST (will take %s)', duration);
    return t_o(duration);
}

function report() {
    console.log('JOB DONE');
}

function done() {
    console.log('ALL WORK DONE');
}

function t_o(delay) {
    return new Promise(function (resolve, reject) {
        setTimeout(function () {
            resolve();
        }, delay);
    });
}

function chooseDelay() {
    var delays = [200, 1000, 4000, 100, 50, 7000];
    return delays[parseInt(Math.random() * 100) % delays.length];
}

var q = new AsyncQueue(work);

for (var packet = 0; packet < 10; ++packet)
    q.enqueue(packet);

q.process(report).then(done);

edited Oct 31 '17 at 23:41

answered Oct 31 '17 at 23:27

Rafael

7,605
13
31
46

This is really nice. Great use of classes and helper functions, very clear. You could work around not returning a promise by wrapping the `this.job()` call with `Promise.resolve()`, but maybe this is not something you want to silently allow, because it likely indicates a bug like forgetting to return the promise, for instance. – GregL Oct 31 '17 at 23:42
Is there any access to the results of *each* call? – Jaromanda X Nov 01 '17 at 00:05
sure, `report` has a `data` param – Rafael Nov 01 '17 at 00:14
To clarify, process() calls `cb` param with `data` arg. – Rafael Nov 01 '17 at 00:21
oh, right, your example function `t_o` resolves undefined :p – Jaromanda X Nov 01 '17 at 00:21
This worked great! I put together a quick example to test it, and had no issues! However, a colleague pointed out that we are using Bluebird.js to handle our Promises, and they have several functions for doing just this sort of thing (specifically `reduce()`, `each()` and `reflect()`), so that's what I actually ended up going with. – Adam Templeton Nov 01 '17 at 19:30

score 1 · Answer 2 · answered Nov 01 '17 at 02:58

As an alternative to an otherwise good answer, here's a really simple queue that works (work function shamelessly copied and expanded from other answer)

    // Here is the queue "creator"
    let promiseQueue = fn => {
        let q = Promise.resolve();
        return (...args) => q = q.then(() => fn(...args));
    };
    // that's it, that's the whole code for a promise queue

    // here we create a queue
    var q = promiseQueue(work);

    // simple mock asynchronous function
    function work({index, data}) {
        var duration = parseInt(Math.random() * 100) + 100;
        console.log('START JOB %s, I.E., MAKE REQUEST (will take %s) and should result with %s', index, duration, (index +1) * data);
        return new Promise(resolve => setTimeout(resolve, duration)) // mock a delay
        .then(() => ({index, result:(index + 1) * data})); // and some "processing"
    }


    // simulating two "chunks" of packets, generated a millisecond apart, but still, the sequence will be maintained
    setTimeout(() => {
        var packets = Array.from({length:10}, (_, index) => ({index, data:parseInt(Math.random() * 10000)}));
        var promises = packets.map(packet => q(packet));
        // the results in promise all are all the results of this batch of "packets"
        Promise.all(promises).then(results => console.log(results));
    }, 100);
    setTimeout(() => {
        var packets = Array.from({length:10}, (_, index) => ({index: index + 10, data:parseInt(Math.random() * 10000)}));
        var promises = packets.map(packet => q(packet));
        Promise.all(promises).then(results => console.log(results));
    }, 101);

Vikash Dahiya · Answer 3 · 2017-11-01T07:11:04.657

the simple function to execute promises sequentially

const sequentiallyExecuting = (promises) => {
    let promise = Promise.resolve();
    promises.forEach((task) => {
        promise = promise.then((data) => {
            return task;
        })
    });

    return promise;
}
// pass array of promises to this function
sequentiallyExecuting(promises).then((data) =>{
   console.log("all requests completed sequentially");
})

score 1 · Answer 4 · answered Nov 01 '17 at 12:09

'use strict';

// job to be done
function job(params) {
 return function () {
  console.log('job started', params);
  return new Promise(function (resolve) {
   setTimeout(function () {
    console.log('job finished');
    resolve();
   }, 1000);
  })
 }
}

// data to be processed sequentially 
var params = [
 1,
 2,
 3,
 4,
 5
];
// reduce data to Promise sequence
params.reduce(function (cum, cur) {
 return cum.then(job(cur));
}, Promise.resolve());

score 0 · Answer 5 · answered Jan 25 '19 at 01:33

0

With async/await it becomes trivial:

while (promiseArray.length > 0)
        await promiseArray.shift();

answered Jan 25 '19 at 01:33

bert

1,556
13
15

resolving Promises sequentially

5 Answers5

Get weird with Promises