Using SharedArrayBuffer with objects

Question

Examples I can find online of using SharedArrayBuffer all seem to be stupid simple, like setting a single byte in the buffer array and then sharing that. But those examples are extremely impractical because no one ever works with raw binary data. No one is every going to do:

myArrayBuffer[0] = 12;

An example of what I'm trying to do:

const sharedArray = new SharedArrayBuffer();
const big64Array = new BigUint64Array(sharedArray);

const entry = {
    Hash: (213874914491n).toString(), // manually toString here because bigint isn't serializable
    BestMove: 10493,
    Depth: 3,
    Score: 41,
    Flag: 0,
};

// I want to put `entry` into big64Array. Ideally, I'd do:
big64Array[0] = BigInt(entry);

// Other things I've tried...
const big64Array = new BigUint64Array(entry); // = [];
const uint8Array = new Uint8Array(entry); // = [];
const uint8ArrayFromText = new Uint8Array(new TextEncoder().encode(JSON.stringify(entry))); // gives an actual byte array

// Apparently this works:
const view = new DataView(uint8ArrayFromText.buffer, 0);
big64Array[0] = view.getBigUint64(0, true);
// But the above seems very...contrived. And I'll need to use a different DataView to convert it back, which I haven't figured out how to do.

Performance is also extremely important here, so if the only way to do this is by calling multiple methods and/or converting a property between various different things this might not work.

Answer 1

Before getting to an example, first I'll discuss a bit of context regarding the details of your question:

When serializing native data (in your question: JS native data types) into a binary format that's in shared memory (and deserializing it somewhere else), you always need to know the number of bytes required by each part of your data structure.

In the details of your question, you show a BigInt and some number types. In JavaScript, a number is always a double-precision 64-bit binary format IEEE 754 value, and when dealing with integers, the max number value is Number.MAX_SAFE_INTEGER (2 ** 53 -1).

When looking at the typed arrays in JavaScript, this max integer value is too large be representable as a single element in any of them except the 64-bit variants. This means that, in order to represent any JS number in each typed array element, you must allocate 64-bits (or 8 bytes) for each number. However: if you are certain that the possible values of a number type will never exceed some smaller limit, then you can store that value more efficiently by allocating less memory (fewer bytes) for it. For example: if the number will never exceed 8 bits (e.g. it's always in the range of 0 to 255 in the case of unsigned integers, or -128 to 127 for singed ints), then you can allocate a single byte for it instead of 8 bytes. Or if it's in the range of a 16-bit number (e.g. 0 to 65535 unsigned, or -32768 to 32767 signed), then you can simply allocate 2 bytes for it.

Because you didn't provide this information in your question, I've made some arbitrary choices for you in the example below, assuming that entry.BestMove will be 16-bit unsigned and the other number types will be 8-bit unsigned. If this is not the case for your actual data, then you can adjust the information accordingly, but I've chosen this to show some variety.

In addition to typed arrays, JS also offers an interface for reading and writing specific number types in a binary ArrayBuffer: the DataView. In the example below, I demonstrate how to use a data view with a struct like the one in your question. I've provided lots of inline comments, but if you're still uncertain about something after reading through it (and all of the links I've provided above), feel free to leave a comment.

'use strict';

// See: https://en.wikipedia.org/wiki/Endianness
// Whether this is set to true or false doesn't really matter —
// it only needs to be applied consistently when accessing the binary slices
// which occupy greater than 1 byte:
const littleEndian = true;

// A map of encoded information about each entry component:
// - its byte length
// - its starting byte index in a binary array buffer
// - a method for writing its associated value to a data view
// - a method for retrieving its associated value from a data view
const entryBinaryMap = {
  Hash: {
    byteLength: 8,
    index: 0, // first index is 0
    set (dataView, bigint) {
      dataView.setBigUint64(this.index, bigint, littleEndian);
    },
    get (dataView) {
      return dataView.getBigUint64(this.index, littleEndian);
    },
  },
  BestMove: {
    byteLength: 2,
    index: 8, // previous index + previous byte length
    set (dataView, number) {
      dataView.setUint16(this.index, number, littleEndian);
    },
    get (dataView) {
      return dataView.getUint16(this.index, littleEndian);
    },
  },
  Depth: {
    byteLength: 1,
    index: 10, // previous index + previous byte length
    set (dataView, number) {
      dataView.setUint8(this.index, number);
    },
    get (dataView) {
      return dataView.getUint8(this.index);
    },
  },
  Score: {
    byteLength: 1,
    index: 11, // previous index + previous byte length
    set (dataView, number) {
      dataView.setUint8(this.index, number);
    },
    get (dataView) {
      return dataView.getUint8(this.index);
    },
  },
  Flag: {
    byteLength: 1,
    index: 12, // previous index + previous byte length
    set (dataView, number) {
      dataView.setUint8(this.index, number);
    },
    get (dataView) {
      return dataView.getUint8(this.index);
    },
  },
  BYTE_LENGTH: 13, // previous index + previous byte length = total bytes
};

// Use the map methods to apply the entry data to the buffer:
function serialize (buffer, entry) {
  const view = new DataView(buffer);
  entryBinaryMap.Hash.set(view, entry.Hash);
  entryBinaryMap.BestMove.set(view, entry.BestMove);
  entryBinaryMap.Depth.set(view, entry.Depth);
  entryBinaryMap.Flag.set(view, entry.Flag);
  entryBinaryMap.Score.set(view, entry.Score);
}

// Get the individual entry values using the map methods:
function deserialize (buffer) {
  const view = new DataView(buffer);
  return {
    Hash: entryBinaryMap.Hash.get(view),
    BestMove: entryBinaryMap.BestMove.get(view),
    Depth: entryBinaryMap.Depth.get(view),
    Flag: entryBinaryMap.Flag.get(view),
    Score: entryBinaryMap.Score.get(view),
  };
}

// Make it even more convenient to use:
function serde (buffer, entry) {
  return entry ? serialize(buffer, entry) : deserialize(buffer);
}

// I'm using a regular ArrayBuffer here because the Stack Overflow code snippet
// sandbox doesn't allow for SharedArrayBuffers, but it works the same with both:
const buffer = new ArrayBuffer(entryBinaryMap.BYTE_LENGTH);
// const buffer = new SharedArrayBuffer(entryBinaryMap.BYTE_LENGTH);

const entry = {
  Hash: 213874914491n,
  BestMove: 10493,
  Depth: 3,
  Score: 41,
  Flag: 0,
};

// Serialize to the buffer:
serde(buffer, entry);

// Deserialize from the buffer:
const deserializedEntry = serde(buffer);

// Validate that the values are equal:
const equal = [
  deserializedEntry.Hash === entry.Hash,
  deserializedEntry.BestMove === entry.BestMove,
  deserializedEntry.Depth === entry.Depth,
  deserializedEntry.Score === entry.Score,
  deserializedEntry.Flag === entry.Flag,
].every(Boolean);

console.log(equal); // true

Note that using the map object in the example above is a bit ceremonial, but I included it to help conspicuously illustrate the relationships in the data since you seem to be learning about working with binary data in JavaScript. There are certainly other approaches to solving this problem.