4

I have an application that generates a lot of data. This data needs to be processed before it's usable (WebGL 3D application). So I created a web worker to do the processing to keep from blocking the rest of the UI.

The problem is that when the data is large enough, the very first call to start the web worker causes the tab in Chrome to crash and display the "Aw, Snap" message. I've set breakpoints at the very beginning of the web worker, and it doesn't even make into the web worker. It seems to crash while trying to clone the data for the postMessage() call.

Here is the general data structure (trying to pass an array of these to a web worker):

function MyClass(id) {
    var count32 = getSize();
    var count16 = count32 * 3;

    this.uint16    = new Uint16Array(count16);  // array buffer 1
    this.float32_a = new Float32Array(count32); // array buffer 2
    this.float32_b = new Float32Array(count32); // array buffer 3


    this.a = id;
    this.b = count32;
    this.c = new MyInnerClass();
    this.d = true;
    this.e = 6;
    this.f = {a: 1, b: 2, c: 3, d: 4};
    this.g = [];
}

function MyInnerClass() {
    this.a = -10;
    this.b = -20;
    this.c = -30;
    this.d =  10
    this.e =  20
    this.f =  30
}

The problem is with array buffers 1, 2 and 3. I have an array of MyClass objects, about 15,000, and about 97% of the array buffers have fewer than 200 elements. But for the remaining 3%, the array buffers have anywhere from 1,000 to 40,000 elements.

Interestingly, if I comment out any 2 of the array buffers, the postMessage() containing the classes will work.

Another interesting point: if I click the "Pause on startup" checkbox in the dev tools in Chrome, the postMessage() will sometimes work, even with all 3 array buffers. Otherwise, it will fail everytime.

Anyone know why this is occurring? I couldn't find docs on the constraints of the data that I might be hitting, or other strange internal weirdness. Otherwise, I'm gonna end up refactoring my code quite a bit to get around this.

DEMO: I created a jsfiddle to demonstrate this: http://jsfiddle.net/GmvyJ/11/

If you change the data max size in this fiddle (to 15,000), please don't save it that way, otherwise you'll never be able to reach the page anymore.

superqd
  • 333
  • 4
  • 12
  • 1
    I am having a similar issue with my script as well, I pass a multidimensional array through a webworker where the webworkers job is to check a json file for updates and if there are any send a message back with that new data. sadly over time I get the aw snap error. – Keleko Sep 25 '13 at 18:43
  • I'm having the exact same problem, my script doesn't have huge arrays though. Instead, I'm calling a worker thread in the "drag" even on google maps and it's doing the exact same thing. I'm assuming it's because its' just spamming the worker threads with new requests when the user drags around. :-\ – Jared Oct 26 '13 at 16:31
  • Got same problem but i'm confused why fails 5th worker not first? – Blackrabbit99 Apr 25 '14 at 10:29
  • Yeah, I never found an answer to this question/problem. Ultimately, I just passed less data to the worker. Just had to figure out a way to do that, since I couldn't find any info that specified the limits/constraints on data passed to the worker. – superqd Apr 25 '14 at 15:52
  • This is no longer an issue on chrome 41 – Erik Johansson Feb 04 '15 at 09:16
  • @ErikJohansson: I still see the issue on Chrome 41 on a Samsung Galaxy Tab 4. – Andy E Apr 07 '15 at 14:00
  • @AndyE did you find any fix ? – Ayman Elarian May 03 '16 at 18:38
  • @Eng I'm afraid not. – Andy E May 04 '16 at 07:47
  • There seems to be an upper limit, which is not surprising, computer always had limited memory. – Walle Cyril Dec 12 '17 at 07:01
  • Has any one found what the upper limit is? – Pablo Jomer Jan 09 '19 at 11:23
  • I do not know of any upper limit. I could not find documentation on it, which I find odd. It might be because the limit is hardware specific. – superqd Jan 10 '19 at 14:44

1 Answers1

2

Have you tried passing data to Worker thread using Transfer of Ownership. That is particularly useful when you are passing large Array Buffers, that will require you to change postMessage call with one extra argument postMessage(ObjectToTransfer,[ObjectToTransfer.buffer,.....]);

This way you can clone entire object but transfer Array Buffers in that object.

Checkout Using transferable objects from a Web Worker

Community
  • 1
  • 1
user3735044
  • 21
  • 2