0

Lets say I have an arrays called

(Arr1,Arr2) 

and the first array has 2000 objects that have the following

{Cyclic:my.parent,string:"imAString",int:10,Position:{bunchofints}}

lets say I wanted to define Arr2 as

for (i in arr1){let arr1[i]= currentObject   arr2.push(currentObject.Position)}

then for sending arr2 to a web worker.

that is the objective and I was wondering how I would go about doing, first send and access the first array then do the other . this and ik there are array buffers that can be defined then a data view can be made from the buffer but I have no idea how you would go about sending what i have above to a worker considering i don't know what is required to send the buffer and how to add the necessary restrictions like byte length. I saw a post that the guy used something like

    var arrayBuffer = new ArrayBuffer("?what should go here");
    var data = new DataView(arrayBuffer);
    var tempArray = new Float32Array(data.byteLength/Float32Array.BYTES_PER_ELEMENT);  

is this on the lines of creating what i want and if not , how can i get to transferring my objects to the worker, Thanks have a blast .

Ps: the reason I made this thread is to simplify the already existing information that I thought was a bit general and unwelcoming to new people.

static660
  • 81
  • 1
  • 13
  • 1
    Don't make a mountain out of a mole hill. – StackSlave Nov 10 '17 at 01:23
  • How so may I ask? – static660 Nov 10 '17 at 01:42
  • What are you actually trying to achieve? – StackSlave Nov 10 '17 at 01:44
  • To simplify the use of the arrayBuffer and transferable objects. – static660 Nov 10 '17 at 01:45
  • How is the question related to `Worker` or transferable objects? – guest271314 Nov 10 '17 at 01:46
  • So you want to convert your data structure to an ArrayBuffer in order to transfer it? Makes no sense. To convert it to an ArrayBuffer, you would have to somehow serialize your object. This serialization means that you will create a copy of it. Then there is no point to *transfer* this serialized version, since you'll still have the original in the main thread. – Kaiido Nov 10 '17 at 01:54
  • What I thought was there was a way of sending the whole array to the worker or from it with just one copy. I thought that's what the transferable objects were. – static660 Nov 10 '17 at 02:17
  • @static660 Is the question how to transfer the array from a thread to `Worker`? – guest271314 Nov 10 '17 at 02:19
  • Indeed and other way around, thanks @guest271314 :) – static660 Nov 10 '17 at 02:20
  • But unfortunately I need to do this in rapid succession, and to do that I was hoping the transferable objects would help – static660 Nov 10 '17 at 02:22
  • Well yes that's what transferable objects are, but you have to work from the transferable object directly for it to make sense. Otherwise, if you do convert a non transferable object every time you want to send it, there is just no point in transferring it. – Kaiido Nov 10 '17 at 02:23
  • How can I do that @kaiido ? – static660 Nov 10 '17 at 02:23
  • Do what? Work from the transferable object? Well ArrayBuffers are transferrables, so you might use it, but this implies that you know in advance the size you'll need it to be. That just depends on your whole app, it is highly probable that it won't fit with your project at all, and it sounds like you are complicating things with this idea. What is the end goal? What should be done on the Worker and what on the main thread with this data? One solution I do use often is to keep all big raw data in Workers and only send computed ones to the main thread but can't know if it's applicable to you. – Kaiido Nov 10 '17 at 02:31
  • I need to send either from the worker A (smaller) yet maybe large number of objects depending on what needs to be rendered at the time (it's a 2d game) , or I can do it the other way and be inefficient as he'll and send 15k objects to narnia, It doesn't matter I wouldn't think about memory allocation as long as you know what the max is right so if I assume always the worst case scenario - – static660 Nov 10 '17 at 02:42
  • I can run the program solely depending on the worst and do it that way. But doing it all on the workers is preferred and I just need a way to send the objects that need to be rendered to the main thread because in the game I'm building their could be lots of bullets and ships going around on screen at a time and I like to prepare for the end before I get there. My modo is to do it right the first time. I learned all ik about js in two months and built a game already, I just need a bit of help understanding js sometimes because it is not at all like other languages. – static660 Nov 10 '17 at 02:42

1 Answers1

0

You can use JSON.stringify() to covert a JavaScript plain object to a string TextEncoder() .encode() method to convert string to a Uint8Array, pass Uint8Array .buffer property to second parameter of postMessage(), then delete the original object defined as a property of window or other object.

At Worker thread use TextDecoder() .decode() method and JSON.parse() to get the transferred object. Define the object as a property of Workerobject within message event.

Perform same task of converting object to string and converting to TypedArray to get ArrayBuffer at .buffer property to transfer ArrayBuffer representation of plain object to Window, delete the property at Worker.

At window

this.data = {Cyclic:123,string:"imAString",int:10,Position:{a:1, b:2, c:3}};

const worker = new Worker("worker.js");    
const encoder = new TextEncoder();    
const decoder = new TextDecoder();

worker.onmessage = e => {
  console.log(e.data)
}

console.log("original data in Window:", this.data);

this.encoded = encoder.encode(JSON.stringify(this.data));

worker.postMessage(this.encoded, [this.encoded.buffer]);

delete this.data;
delete this.encoded;

console.log("data deleted in Window:", this.data, this.encoded);

worker.onmessage = e => {
  this.data = JSON.parse(decoder.decode(e.data));
  console.log("data in message event at window:", this.data);
}

at Worker

const decoder = new TextDecoder();
const encoder = new TextEncoder();

self.onmessage = e => {
  console.log(e.data);
  self.data = JSON.parse(decoder.decode(e.data));
  self.data.string = "imAStringToo";
  self.encoded = encoder.encode(JSON.stringify(self.data));
  console.log("data in Worker:", self.data);
  self.postMessage(self.encoded, [self.encoded.buffer]);
  delete self.data; 
  delete self.encoded;
  console.log("data deleted in Worker:", self.data, self.encoded);
}

plnkr http://plnkr.co/edit/my7GS0XRC5iEwjL0I2th?p=preview

guest271314
  • 1
  • 15
  • 104
  • 177
  • So now instead of allocating twice the memory needed for the data, you allocate it four times... Ps: typo in `Uint9Array`. – Kaiido Nov 10 '17 at 03:03
  • @Kaiido _"you allocate it four times"_ Can you provide details for what you are describing? – guest271314 Nov 10 '17 at 03:24
  • You're right it might be more... **Step1** `var data = {hello: 'world'}` => allocated once. **Step2**: `str = JSON.stringify(data)` => allocated twice. **Step 3**: `buf = await TextEncoder.encode(str)` => allocated thrice. **Step 4**: `transfer(buf)` => no more allocation. **Step 5**: `out_str = await TextDecoder.decode(buf)` => allocated 4 times. **Step 6**: `out_data = JSON.parse(out_str)` => allocated 5 times. – Kaiido Nov 10 '17 at 03:33
  • @Kaiido https://stackoverflow.com/questions/1947995/when-should-i-use-delete-vs-setting-elements-to-null-in-javascript – guest271314 Nov 10 '17 at 03:37
  • And ? What's the point here? You could just as well mark the only copy in main thread to GC after postMessage did the swallow copy on its own. The whole point of transferables are to avoid allocation of the same data, the memory slots are given to the other side. There is no point at all in doing this if you do allocate duplicate memory anyway. – Kaiido Nov 10 '17 at 03:42
  • @Kaiido Not entirely sure what you are trying to convey? Do not try to convert a plain object to a transferable object? Use a different data structure to avoid `JSON` method calls and `delete`? Pass the plain object to `postMessage()` without transferring the object, to create only a single copy of the object at `Worker` thread? – guest271314 Nov 10 '17 at 03:47
  • 1
    Do not try to make something transferrable if there is no point in doing so? Yes using raw postMessage method would be beneficial here. – Kaiido Nov 10 '17 at 03:49
  • @Kaiido Fair enough. Now OP should be able to gather how to transfer a plain object, reasons why the transfer process might impact memory and that passing the plain object to `postMessage()` is also a viable option, and make appropriate decision as to how to proceed with the code. – guest271314 Nov 10 '17 at 03:52
  • In the end, the true answer should be using an array buffer example with the arrays supplied in a format that people can understand (via step) this is my problem but the post is about other people as well and if we can solve this then others may be able to follow by example. – static660 Nov 10 '17 at 08:03
  • Thanks @guest271314 for supplying a relevant code and in a proper format, all it could use are maybe some pointers via comments, other than that kaiido has made it clear in my particular case it won't work. kaiido thanks as usual for the helpful pointers, if you know of the most effective way to send objects from a worker to the main thread to render them that would be just helpful for me personally, and may even answer another thread that pops up. Or hell I'll probably make a tutorial on all this when I'm done what I'm doing. Thanks guys :) – static660 Nov 10 '17 at 08:10