23

Is there some way to synchronously wait for or check for a new message in a web-worker?

I have a large complicated body of code (compiled LLVM from emscripten) that I cannot refactor around callbacks.

I need to make sure that code after a certain line doesn't execute until I receive and handle a message from the UI-thread. If I block with a while-loop, the event-loop never runs so I can't receive messages.

alex
  • 479,566
  • 201
  • 878
  • 984
bkase
  • 337
  • 3
  • 9
  • 1
    There would be a solution to your problem if ECMAScript was a language as rich as Scheme with its `call-with-continuation`. This would allow you to catch the current continuation and call it again in your `onmessage` handler. – Marc Jan 05 '15 at 11:55
  • 1
    It is now possible with the Atomics API or by abusing a service worker https://glitch.com/edit/#!/sleep-sw?path=worker.js%3A28%3A71 – Stefnotch Jul 18 '21 at 10:42
  • 1
    @Stefnotch the SW hack is quite clever, but to really do what OP wanted you'd need to pass the data through the response. They can't access the Worker's message-queue because *their* `burnCpu` never comes back to the event-loop (no setTimeout). However from your SW you could wait that the main thread posts a message to the SW (instead of waiting the time param), and pass that as the response to the sync XHR. Though you'd be limited to Strings, ArrayBuffers and Blobs, and would be copying the two last ones instead of just linking to them... but might be a good answer here anyway. – Kaiido Aug 27 '21 at 01:19

5 Answers5

9

This is an issue that I also ran into while working with Pyodide. I wanted to 'synchronously' call a function from the main thread.

One solution involves the Atomics and SharedArrayBuffer APIs. From the perspective of the web worker, this looks like the following

  1. postMessage the main thread
  2. freeze ourselves with Atomics.wait
  3. get unfrozen by the main thread
  4. read the result from a SharedArrayBuffer. We can't receive the result as a postMessage, because there isn't a synchronous way of asking "did I get a message".

Of course this requires a fair amount of extra code to handle all the serialization, data passing and more.

The main limitation is that to use those APIs, one needs to have the COOP/COEP headers set. The other bit to keep in mind is that this only works on recent browsers such as Safari 15.2 (released in December 2021).

There is also an alternative solution with synchronous XHR and a service worker, but I haven't investigated that option.

Stefnotch
  • 511
  • 2
  • 13
  • 26
  • 1
    You could probably use the SharedArrayBuffer itself to pass some data from the main thread. See https://github.com/GoogleChromeLabs/buffer-backed-object for some ideas on how to serialize most objects. – Kaiido Aug 28 '21 at 09:01
  • This is not a good answer. The answer by Pasha Bolokhov https://stackoverflow.com/a/68945416/857427 is in fact the correct one. I don't know why it hasn't received enough attention. – arashka Jul 20 '23 at 16:16
  • 1
    @arashka because they don't answer the question as it's being asked. Maybe it solves *your* issue, but it won't solve the one presented in the original post here. – Kaiido Jul 21 '23 at 03:32
  • @Kaiido Please read my comment on your other comment on the answer by Pasha Bolokhov (https://stackoverflow.com/a/68945416/857427). You are misinterpreting the question. As this user asks, he wants to make sure a part of code is executed only after another (I need to make sure that code after a certain line doesn't execute until I receive and handle a message from the UI-thread). This doesn't refer to the whole runtime, this is about a code block, as we usually consider when we speak about synchronous execution model. So async/await means exactly synchronous in this context. – arashka Jul 24 '23 at 06:03
  • No it does refer to the runtime since otherwise they would have been able to refactor it to use callbacks. They want to "*synchronously wait for or check for a new message in a web-worker*". Using `async` is obviously not the solution, it's in the name. – Kaiido Jul 24 '23 at 06:16
2

A crude hack to do this could be to use a synchronous API, such as the FileSystem API, and use a temporary file as a synchronising mechanism.

In your example, the emscripten code could issue a synchronous read on a file, while the UI thread writes to the same file, effectively unblocking the worker.

HRJ
  • 17,079
  • 11
  • 56
  • 80
1

No, unfortunately. There was some discussion of adding a way to block on a message from the parent page, but it went nowhere. I'll see if I can dig up the relevant mailing list thread.

lawnsea
  • 6,463
  • 1
  • 24
  • 19
1

Can you break the sequence as follow without refactoring?

wrk.onmessage = function(e) {
    if (e.data.step === 'initial input') {
        doInitialWork(e.data.inputs)
    } else if (e.data.step === 'ui input') {
        doTheRest(e.data.uiData)
    }
}

The onmessage event will be blocked until the initial step execution stack is complete and the critical ui bits will only launch when and if the info is available.

Hurelu
  • 1,458
  • 1
  • 14
  • 23
  • 1
    This isn't truly blocking. You still end up with two function calls. For example, a blocking web worker would be able to implement a function which takes a forEach style function and converts it into an iterator automatically. – Dyllon Gagnier Apr 24 '21 at 18:53
1

This is how I've done this. It's really sorrowful to think that Web Workers haven't received the await message functionality, which would make the whole inter-thread communication concept a lot closer to Go language for example. But no, they ain't did it, so have to invent stuff in order to keep threads synchronised

  let resolver: any = undefined;
  let message: any = undefined;
  const messagePromise = new Promise((resolve) => resolver = resolve);

  worker.onmessage = async ({ data: payload }: { data: any }) => {
    message = payload;
    worker.onmessage = undefined;
    resolver?.();
  }

  await messagePromise;

Now, this is Typescript. Just remove the types to get Javascript. Note that there is no way to capture a message if it had been sent before we started listening for messages

Pasha Bolokhov
  • 56
  • 2
  • This is not really what OP asked for. Their case was that they can't break their code to run on multiple tasks, i.e it can't let the event-loop actually loop and process its queued messages. If you wish a Promise based messaging system between your UI context and the Worker one, then have a look at MessageChannels which will allow your code to create one-off channels on which you'll be sure to be the only one to listen to. Here is an example: https://stackoverflow.com/questions/62076325/how-to-let-a-webworker-do-multiple-tasks-simultaneously/62077379#62077379 – Kaiido Aug 27 '21 at 01:25
  • Message channels are not blocking either. There's no built in way to say "I want to wait until I get a message" – Pasha Bolokhov Aug 28 '21 at 19:06
  • No but it helps doing what you are trying to do: promisify the communication – Kaiido Aug 29 '21 at 00:11
  • No, I'm trying to synchronize threads, like it would happen in Go – Pasha Bolokhov Aug 30 '21 at 02:39
  • But the code in your answer is just creating a promisified communication system, except that it won't handle well multiplexing. – Kaiido Aug 30 '21 at 02:44
  • I don't understand what we are arguing about. The code above is far from being a complete synchronisation tool for threads. The code is the answer to — what I thought was — the original question on this page, whether it asnwers it or not. Yes you are right it does not handle multiplexing, and it does create a promise as a natural way of suspending execution in Javascript. What I do _need_ is a different question — existence of blocking communication channels. So you are right, and also I am right in that I do need synchronisation. So what's the arguing about? – Pasha Bolokhov Aug 31 '21 at 05:25
  • Well, you did post this code as an answer to "how to **synchronously** receive a message from outside". Using `async` is obviously not a correct answer to that question. So while I was at pointing that out to you, I also wanted to be nice and point out that even to do what this code is doing, i.e promisifying the communication, which is a common problem too even if not the one we're interested in this Q/A, there are better ways to do so. – Kaiido Aug 31 '21 at 05:32
  • @Kaiido You are wrong, this answer is in fact correct. The function is async but the await makes it synchronous and this is the only way to do it. – arashka Jul 20 '23 at 16:14
  • @arashka nope. The code is asynchronous, the event loop will perform other tasks while the worker is doing its stuff. – Kaiido Jul 20 '23 at 23:05
  • @Kaiido You misunderstand synchronous execution model. When we speak about synchronous execution model, we are referring to a code block, not the whole system. Using async and await causes the code block to run synchronously. – arashka Jul 24 '23 at 06:01
  • @arashka No I don't misunderstand anything here, and no it doesn't run synchronously. OP wants that when they run the code on the Worker side, no other task can run on the main thread until they get the result. At every `await somethingAsync` the event loop will pause the current script execution, do other stuff like executing other tasks, and then eventually resume the execution of the script. This is basically what we call asynchronicity. – Kaiido Jul 24 '23 at 06:14
  • @Kaiido Please try to read my comments before answering. Instead of being pedantic, try to see what other people are saying. Read the question again. You just answer and reiterate your point, but in fact what you say is not related to the question. The person wants to make sure a part of his code runs after another part, this is what he says, that's it. – arashka Jul 25 '23 at 10:16