1

I'm using a websocket API that streams real time changes on a remote database, which I then reflect on a local database. The operations on my side must be done in the same order (I create and update records as the data comes in).

My problem is the messages often arrive in bulk, faster than I can process them and end up out of order. Is there a way to make onmessage wait before I'm ready to process the next one ?

Here's an example of what I'm trying, without sucess:

async doSomething(ev) {
    return new Promise(async (resolve, reject) => {
        // database operations, etc.
        resolve(true);
    });
}

ws.onmessage = async (ev) => {
    await doSomething();
}
Vaneik
  • 2,963
  • 1
  • 12
  • 9

1 Answers1

0

You can create a queue (e.g., an array of values) and use Promises (and possibly and AsyncIterator, or ReadableStream, if needed) to process the data in sequential order.

For example

new ReadableStream({
  pull(controller) {
    controller.enqueue(event.data)
  }
})

// ..

reader.read()
.then(({value, done}) => {
  // do stuff
})
guest271314
  • 1
  • 15
  • 104
  • 177
  • @Vaneik See [Receiving data via stdin and storing as a variable/array](https://stackoverflow.com/q/45427642/); [HTML5 audio streaming: precisely measure latency?](https://stackoverflow.com/questions/38768375/html5-audio-streaming-precisely-measure-latency/); [Chrome memory issue - File API + AngularJS](https://stackoverflow.com/questions/41440235/chrome-memory-issue-file-api-angularjs); [Node.js: splitting stream content for n-parts](https://stackoverflow.com/questions/43631320/); [Run multiple recursive Promises and break when requested](https://stackoverflow.com/q/48349721/) – guest271314 Feb 16 '19 at 01:37
  • @Vaneik See also [Promises for promises that are yet to be created without using the deferred \[anti\]pattern](https://stackoverflow.com/q/37426037/) – guest271314 Feb 16 '19 at 01:42
  • Yep, I ended up doing a queue. – Vaneik Feb 25 '19 at 23:18
  • 1
    @Vaneik do you mind sharing how you did it exactly? – dikokob Dec 23 '20 at 07:43
  • I've since retired that code, but it was straightforward: top priority was receiving the data and adding it to a queue in the order received. Then periodically process the queue and store the data. It worked flawlessy for several months. – Vaneik Dec 24 '20 at 11:57