How to wait for multiple WebWorkers in a loop

Question

I have the following issue with Web Workers in JS. I have a heavy duty application doing some simulation. The code runs in multiple Web Workers. The main thread is running on a WebPage. But could also be a Web Worker, if it makes sense.

Example:

    var myWebWorkers = [];
    function openWorker(workerCount){
        for(var i = 0; i < workerCount; i++){
            myWebWorkers[i] = new Worker('worker.js');
            myWebWorkers[i].onmessage = function(e){
                this.result = e.data;
                this.isReady = true;
            }
        }
    }

    function setWorkerData(somedata){
        // somedata.length is always a multiple of myWebWorkers.length
        var elementCntPerWorker = somedata.length / myWebWorkers.length;
        myWebWorkers.forEach(function(worker, index){
            worker.isReady = false;
            worker.postMessage(
                somedata.slice(index * elementCntPerWorker, 
                    (index + 1) * elementCntPerWorker - 1));
        });
    }

    var somedata = [...];
    openWorker(8); 
    for(var i = 0; i < 10000; i++){
         setWorkerData(somedata);
         waitUntilWorkersAreDoneButAllowBrowserToReact(myWebWorkers);
         if(x % 100) updateSVGonWebPage
    }

    function waitUntilWorkersAreDoneButAllowBrowserToReact(){
        /* wait for all myWebWorkers-onchange event, but
           allow browser to react and don't block a full Web Worker 
           Following example is my intension. But will not work, because 
           events are not executed until code excution stops.
        */
        somedata = [];
        for(var i = 0; i < myWebWorkers.length; i++){
            while(!myWebWorkers[i].isReady);
            somedata = somedata.concat(myWebWorkers.result);
        }
    }

What I need is really the waitUntilWorkersAreDoneButAllowBrowserToReact function or a concept to get this running. Every searching reagarding Mutex, sleep, etc ends in the following sentences: "JS is single threaded", "This will only work if you are not in a loop", "There is no reason to have a sleep function". etc.

Even when passing the main task to another Worker, I got the problem, that this thread is 100 % duty on checking, if the others are ready, which is waste of energy and processing power.

I would love to have a blocking function like myWebWorker.waitForReady(), which would allow events still to be handled. This would bring javascript to its next level. But may be I missed a simple concept that will do exactly this.

Thank you!

Are you trying to call `waitUntilWorkersAreDoneButAllowBrowserToReact` when all workers which have received message at `setWorkerData(somedata)` calls have responded with a message to main thread? That is, when `for` loop completes call `waitUntilWorkersAreDoneButAllowBrowserToReact`? Difficult to interpret exactly what you are trying to accomplish? — guest271314, Sep 01 '16 at 22:02
Why are there no `message` event handlers at example `javascript` at Question? — guest271314, Sep 01 '16 at 22:08
@guest271314: I've added more code to make it easier to understand — Frank, Sep 01 '16 at 22:36
`while` loop within `for` loop freezes browser? Does web worker have `change` event? Are you trying to call a function when all web workers have posted a message to main thread? — guest271314, Sep 01 '16 at 22:47
Yes try `while(true)` in a console. Corrected above to onmessage. — Frank, Sep 01 '16 at 22:51
You can substitute using `Promise.all()` or other function for `while` loop to call a function when all web workers have posted at least one message to main thread — guest271314, Sep 01 '16 at 22:53
Please have in mind, that I simplified this example. As it is in the original version two loops, with different status variables and abort functionility. The intention of this question is really to get a concept for the WebWorkers, not to have running code. This is not debugging, as I have a blocking WebWorker version. — Frank, Sep 01 '16 at 22:54
_"The intention of this question is really to get a concept for the WebWorkers, not to have running code."_ "get a concept for the WebWorkers" can be broadly interpreted. Not certain what expected Answer is? See http://stackoverflow.com/help/how-to-ask — guest271314, Sep 01 '16 at 22:58
Ok, as this is already asychronous due to the event `onmessage`. I could easily do a `for` loop in the onmessage event, to check all `isReady` flags, and if fullfilled, catch the new data in `somedate`, and set a global variable. However, this still doesn't allow me to use a `for` loop in the main thread. It is the same with the `Promise` concept. As `Promise` is non blocking and uses callbacks. I'm wrong? — Frank, Sep 01 '16 at 23:05
There are probably several approaches that could be used to call a function only when _n_ variables have a value of `true`. `while` loop with `for` loop could block UI and freeze browser. — guest271314, Sep 01 '16 at 23:09

score 6 · Accepted Answer · edited May 23 '17 at 12:22

I would love to have a blocking function like myWebWorker.waitForReady()

No, that's not possible. All the statements you researched are correct, web workers stay asynchronous and will only communicate by messages. There is no waiting for events, not even on worker threads.

You will want to use promises for this:

function createWorkers(workerCount, src) {
    var workers = new Array(workerCount);
    for (var i = 0; i < workerCount; i++) {
        workers[i] = new Worker(src);
    }
    return workers;
}
function doWork(worker, data) {
    return new Promise(function(resolve, reject) {
        worker.onmessage = resolve;
        worker.postMessage(data);
    });
}
function doDistributedWork(workers, data) {
    // data size is always a multiple of the number of workers
    var elementsPerWorker = data.length / workers.length;
    return Promise.all(workers.map(function(worker, index) {
        var start = index * elementsPerWorker;
        return doWork(worker, data.slice(start, start+elementsPerWorker));
    }));
}

var myWebWorkers = createWorkers(8, 'worker.js');
var somedata = [...];
function step(i) {
    if (i <= 0)
        return Promise.resolve("done!");
    return doDistributedWork(myWebWorkers, somedata)
    .then(function(results) {
        if (i % 100)
            updateSVGonWebPage();
        return step(i-1)
    });
}
step(1000).then(console.log);

Promise.all does the magic of waiting for concurrently running results, and the step function does the asynchronous looping using a recursive approach.

How to wait for multiple WebWorkers in a loop

1 Answers1

Linked