Can I encode binary JS data - including uIntArrays and ArrayBuffers - to a string format in a single step?

Question

Similar to Binary Data in JSON String. Something better than Base64 however I'm asking about JS datatypes specifically (like Uint8Array) and don't have a requirement to use JSON.

I have a JS Object with many nested values including values of type Uint8Array and ArrayBuffer that I need to send as an encoded string.

I know I can convert individual fields to strings and then manually decode each specific field on the other end.

That's the approach I'm currently taking - through code like:


export const uInt8ArrayToString = (uInt8Array: Uint8Array) => {
  const string: string = utf8Decoder.decode(uInt8Array);
  return string;
};

export const arrayBufferToString = (buffer: ArrayBuffer): string => {
  return utf8Decoder.decode(new Uint8Array(buffer));
};

And for decoding:

export const stringToUint8Array = (string: string) => {
  const uint8Array: Uint8Array = utf8Encoder.encode(string);
  return uint8Array;
};

export const stringToArrayBuffer = (string: string) => {
  const uint8Array: Uint8Array = utf8Encoder.encode(string);
  const arrayBuffer: ArrayBuffer = uInt8ArrayToArrayBuffer(uint8Array);
  return arrayBuffer;
};

However this seems somewhat inefficient, as I have to note the data type used by each field and encode/decode specific fields on end/

Can I encode binary JS data - including uIntArrays and ArrayBuffers - to a string format in a single step?

Look into JSON.stringify and parse. In both you can add a replacer and reviver function where you can do custom stringification/parsing based on type of object. Details in https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/JSON/stringify#using_a_function_as_replacer and https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/JSON/parse#using_reviver_when_paired_with_the_replacer_of_json.stringify — Peter Thoeny, Mar 11 '23 at 00:31
Thanks @PeterThoeny! I did consider this but rather than have a bunch of my own functions, considered that there would be other JS user that needed to store ArrayBuffers and uInt8Arrays etc. and eventually found esserializer. — mikemaccana, Mar 12 '23 at 13:14

score 0 · Accepted Answer · answered Mar 12 '23 at 13:06

Answering my own question to help others

Looking at other questions (which were mainly focused on high-level types like class instances rather than uIntArrays and ArrayBuffers) I was able to find esserializer

Run with npx esrun runme.js:

import ESSerializer from "esserializer";
import { log } from "console";

const foo = {
  bar: new Uint8Array([1, 2, 3]),
  baz: new ArrayBuffer([4, 5, 6]),
};

const serialized = ESSerializer.serialize(foo);

const deserialized = ESSerializer.deserialize(serialized);

log(deserialized);

Will return:

{
  bar: Uint8Array(3) [ 1, 2, 3 ],
  baz: ArrayBuffer { [Uint8Contents]: <>, byteLength: 0 }
}

Can I encode binary JS data - including uIntArrays and ArrayBuffers - to a string format in a single step?

1 Answers1