3

I am passing an empty array to a worker, which should populate the array and return it - I know there are other ways of doing this, but I'm more interested in why it isnt working, than getting it to work.

The main code:

var arr = new Array(4)
console.log(arr.length)//outputs 4
var worker = new Worker("whatever.js")
worker.postMessage(arr)

The worker code

self.onmessage = function(msg){
    console.log(msg.data.length)//outputs 0
}

If I pass in a populated array, it works. If I even set a single entry of the array to a value, it works.

Why does the postMessage function collapse non-zero-length, but empty arrays, is there a way of avoiding this? (apart from manually assigning a value) .

I'm using Windows 7 and Chrome 51

Tomáš Zato
  • 50,171
  • 52
  • 268
  • 778
Zack Newsham
  • 2,810
  • 1
  • 23
  • 43

1 Answers1

0

Data is copied to workers via the structured cloning algorithm, so your worker is not getting the exact array you created, that would be a thread-unsafe operation as arrays are a reference type. This is a case where the MDN docs are incomplete, generally speaking with a few exceptions the structured cloning algoritm is equivalent to JSON.parse(JSON.stringify(x)) but this is an exception as JSON.parse(JSON.stringify(new Array(4))) yields [null, null, null, null] at least in chrome and ff.

The weird behavior here is probably in part because there really isn't any point to creating arrays with empty slots like that, there aren't even any indicies, e.g. new Array(4).forEach(i => console.log('foo')); doesn't do anything. So the structured cloning algorithm generates an empty array.

Note that the structured cloning algorithm is not part of the JavaScript specification, it is rather part of the HTML 5 spec and as far as I can tell doesn't give a lot of details, so I'm not entirely sure how it works, but seems to be focused on things like fileData and Blobs. Note that cloning an arraybuffer is part of the Ecmascript spec. All of that makes some sense, workers have a huge performance penalty for the communication overhead (which is why shared-memory constructs have been proposed) in no small part due to that same algorithm. So the computation you want to do in them has to be intensive enough to outweigh the startup and communication penalties. So it makes sense that the communication channel is more focused on those low-level (essentially binary in the blob case) data constructs.

Jared Smith
  • 19,721
  • 5
  • 45
  • 83
  • thanks for the reply - I'm using a library called Slang, it uses that pattern to return a repeated string, e.g., new Array(4).join("1") will return "1 1 1", its used similarly elsewhere too, which is where I need to pass it to a worker. As I said in my question, I'm more interested in knowing why it isnt being copied in as defined, rather than finding a "fix" – Zack Newsham Aug 04 '16 at 14:49
  • @ZackNewsham again, there really isn't any point to doing that, `String.prototype.repeat` is a standard and while fairly recent is trivial to polyfill. I'd chalk it up to weirdness and use more common patterns. Interesting edge case you found though! – Jared Smith Aug 04 '16 at 14:54
  • Good to know about the `String.prototype.repeat` Unfortunately I'm working on a project that requires me to convert some libraries into workers - its a long story - so I can't really change the code under test unless I have to. In regards to it being pointless, I thought the point of it was to allocate an array of the correct size immediately rather than repeatedly resizing using `Array.prototype.push` - I know in this case it makes little difference since its copied into the worker anyway – Zack Newsham Aug 04 '16 at 15:07
  • 1
    @ZackNewsham that's entirely unnecessary. Your JS 'arrays', AFAIK are on the heap anyways, growing them dynamically is no big deal. Not to mention that if your JavaScript isn't IO-bound, why you're doing that much processing client-side? Unless you're writing a browser-based image editor or game console emulator its a broken pencil. Pointless. Its arguably a mistake in the language, `new Array(4)` *should* return `[4]`. `new Array(1,2)` *does* return `[1,2]`. The library you reference is simply an instance of Carmack's Law. http://mdjnewman.me/2013/10/john-carmack-on-type-systems/ – Jared Smith Aug 04 '16 at 15:20
  • Growing arrays on the heap is a big deal, because you may have to reallocate the entire array, to grow it. It is much faster to assign a value to an existing array than it is to push, about 5X in my setup. Try: `var start = new Date().getTime(); var arr = [];for(var i = 0; i < 10000000; i++){arr.push(i)}console.log(new Date().getTime() - start)` vs `var start = new Date().getTime(); var arr = new Array(10000000);for(var i = 0; i < 10000000; i++){arr[i] = i}console.log(new Date().getTime() - start)` obviously this only matters for large arrays – Zack Newsham Aug 04 '16 at 15:40
  • 1
    @ZackNewsham so what? Most of the built in array methods (slice, map, concat, etc.) *copy* the array. Passing it to/from a worker *copies* the array. So your version takes half as many *micro*seconds? Who cares? Is this a hard real-time system? Are you allocating a 1000000+ element array? If so, why? If not, go with the version that is readable and doesn't rely on a mistake in the language. The `with` statement may be fast, but that won't make me use it. – Jared Smith Aug 04 '16 at 15:47
  • ok, we have gotten off topic here, but anywho. First, Javascript isn't purely client side. Second, running code on the client to avoid sending large chunks of data to the server isnt 'bad' - think of the transmission overhead, not to mention privacy. So allocating a large array isn't that uncommon, third its 5 times faster, not "half the microseconds". Fourth, neither is more or less readable, they are virtually the same piece of code. Fifth, its not a language mistake, its part of the specification - presumably for this reason. – Zack Newsham Aug 04 '16 at 15:50
  • Let us [continue this discussion in chat](http://chat.stackoverflow.com/rooms/120146/discussion-between-jared-smith-and-zack-newsham). – Jared Smith Aug 04 '16 at 15:50