How do I merge an array of Uint8Arrays?

Question

I have an array of several Uint8Arrays. Something similar to this:

[Uint8Array(16384), Uint8Array(16384), Uint8Array(16384), Uint8Array(16384), Uint8Array(16384), Uint8Array(8868)]

How do I merge/concatenate/join (not sure what the right term is) them to a single ArrayBuffer?

The key here is that the output I need must be an ArrayBuffer.

Answer 1

You can use the set method. Create a new typed array with all the sizes. Example:

var arrayOne = new Uint8Array([2,4,8]);
var arrayTwo = new Uint8Array([16,32,64]);

var mergedArray = new Uint8Array(arrayOne.length + arrayTwo.length);
mergedArray.set(arrayOne);
mergedArray.set(arrayTwo, arrayOne.length);

Alternative: Convert your typed array in "normal" arrays. concat it and create a type array of it again.

In your case (solution):

let myArrays = [new Uint8Array(16384), new Uint8Array(16384), new Uint8Array(16384), new Uint8Array(16384), new Uint8Array(16384), new Uint8Array(8868)];

// Get the total length of all arrays.
let length = 0;
myArrays.forEach(item => {
  length += item.length;
});

// Create a new array with total length and merge all source arrays.
let mergedArray = new Uint8Array(length);
let offset = 0;
myArrays.forEach(item => {
  mergedArray.set(item, offset);
  offset += item.length;
});

// Should print an array with length 90788 (5x 16384 + 8868 your source arrays)
console.log(mergedArray);

Answer 2

Just treat it like any normal array:

var concatArray = new Uint8Array([ ...Uint8Array1, ...Uint8Array2, ...Uint8Array3 ]);

Answer 3

There has an easy way:

await new Blob([a1,a2,a3]).arraybuffer();

but slow then:

new Uint8Array(a1.length + a2.length + a3.length)
..set(a1,0)
..set(a2,a1.length)
..set(a3,a1.length+a2.length)

Answer 4

You could have a helper function to concatenate an array of arrays of any length, in their provided order.

This looks nice and clean but it also makes copy of acc on each iteration because of the spread operator:

// arrays are Uint8Array[]
(arrays) => new Uint8Array(arrays.reduce((acc, curr) => [...acc, ...curr], []));

This looks a little longer, but is more performant since acc is not spread on each iteration, but pushed to and returned:

// arrays are Uint8Array[]
(arrays) => {
  const flatNumberArray = arrays.reduce((acc, curr) => {
    acc.push(...curr);
    return acc;
  }, []);

  return new Uint8Array(flatNumberArray);
};

Answer 5

This works for me: single_array is what you want

 var arrays = [Uint8Array(16384), Uint8Array(16384), Uint8Array(16384), Uint8Array(16384), Uint8Array(16384), Uint8Array(8868)]



 var single_array = concat(arrays)


function concat(arrays) {
  // sum of individual array lengths
  let totalLength = arrays.reduce((acc, value) => acc + value.length, 0);

  if (!arrays.length) return null;

   let result = new Uint8Array(totalLength);

      // for each array - copy it over result
      // next array is copied right after the previous one
      let length = 0;
      for(let array of arrays) {
            result.set(array, length);
            length += array.length;
      }

      return result;
   }

Answer 6

Function based on Dominik's answer:

function mergeUInt8Arrays(a1: Uint8Array, a2: Uint8Array): Uint8Array {
  // sum of individual array lengths
  var mergedArray = new Uint8Array(a1.length + a2.length);
  mergedArray.set(a1);
  mergedArray.set(a2, a1.length);
  return mergedArray;
}

Answer 7

By extending @Domske approach, I did create a function that merge N Uint8Arrays into a single one:

function mergeUint8Arrays(...arrays) {
  const totalSize = arrays.reduce((acc, e) => acc + e.length, 0);
  const merged = new Uint8Array(totalSize);

  arrays.forEach((array, i, arrays) => {
    const offset = arrays.slice(0, i).reduce((acc, e) => acc + e.length, 0);
    merged.set(array, offset);
  });

  return merged;
}

Typescript:

function mergeUint8Arrays(...arrays: Uint8Array[]): Uint8Array {
  const totalSize = arrays.reduce((acc, e) => acc + e.length, 0);
  const merged = new Uint8Array(totalSize);

  arrays.forEach((array, i, arrays) => {
    const offset = arrays.slice(0, i).reduce((acc, e) => acc + e.length, 0);
    merged.set(array, offset);
  });

  return merged;
}

---

Usage:

const arrayOne = new Uint8Array([2,4,8]);
const arrayTwo = new Uint8Array([16,32,64]);

mergeUint8Arrays(arrayOne, arrayTwo); // Uint8Array(6) [2, 4, 8, 16, 32, 64]