Javascript Typed Arrays and Endianness

Question

I'm using WebGL to render a binary encoded mesh file. The binary file is written out in big-endian format (I can verify this by opening the file in a hex editor, or viewing the network traffic using fiddler). When I try to read the binary response using a Float32Array or Int32Array, the binary is interpreted as little-endian and my values are wrong:

// Interpret first 32bits in buffer as an int
var wrongValue = new Int32Array(binaryArrayBuffer)[0];

I can't find any references to the default endianness of typed arrays in http://www.khronos.org/registry/typedarray/specs/latest/ so I'm wondering what's the deal? Should I assume that all binary data should be little-endian when reading using typed arrays?

To get around the problem I can use a DataView object (discussed in the previous link) and call:

// Interpret first 32bits in buffer as an int
var correctValue = new DataView(binaryArrayBuffer).getInt32(0);

The DataView functions such as "getInt32" read big-endian values by default.

(Note: I've tested using Google Chrome 15 and Firefox 8 and they both behave the same way)

Answer 1

The current behaviour, is determined by the endianness of the underlying hardware. As almost all desktop computers are x86, this means little-endian. Most ARM OSes use little-endian mode (ARM processors are bi-endian and thus can operate in either).

The reason why this is somewhat sad is the fact that it means almost nobody will test whether their code works on big-endian hardware, hurting what does, and the fact that the entire web platform was designed around code working uniformly across implementations and platforms, which this breaks.

Answer 2

FYI you can use the following javascript function to determine the endianness of the machine, after which you can pass an appropriately formatted file to the client (you can store two versions of the file on server, big endian and little endian):

function checkEndian() {
    var arrayBuffer = new ArrayBuffer(2);
    var uint8Array = new Uint8Array(arrayBuffer);
    var uint16array = new Uint16Array(arrayBuffer);
    uint8Array[0] = 0xAA; // set first byte
    uint8Array[1] = 0xBB; // set second byte
    if(uint16array[0] === 0xBBAA) return "little endian";
    if(uint16array[0] === 0xAABB) return "big endian";
    else throw new Error("Something crazy just happened");
}

In your case you will probably have to either recreate the file in little endian, or run through the entire data structure to make it little endian. Using a twist of the above method you can swap endianness on the fly (not really recommended and only makes sense if the entire structure is the same tightly packed types, in reality you can create a stub function that swaps bytes as needed):

function swapBytes(buf, size) {
    var bytes = new Uint8Array(buf);
    var len = bytes.length;
    var holder;

    if (size == 'WORD') {
        // 16 bit
        for (var i = 0; i<len; i+=2) {
            holder = bytes[i];
            bytes[i] = bytes[i+1];
            bytes[i+1] = holder;
        }
    } else if (size == 'DWORD') {
        // 32 bit
        for (var i = 0; i<len; i+=4) {
            holder = bytes[i];
            bytes[i] = bytes[i+3];
            bytes[i+3] = holder;
            holder = bytes[i+1];
            bytes[i+1] = bytes[i+2];
            bytes[i+2] = holder;
        }
    }
}

Answer 3

Taken from here http://www.khronos.org/registry/typedarray/specs/latest/ (when that spec is fully implemented) you can use:

new DataView(binaryArrayBuffer).getInt32(0, true) // For little endian
new DataView(binaryArrayBuffer).getInt32(0, false) // For big endian

However, if you can't use those method because they aren't implemented, you can always check the file's magic value (almost every format has a magic value) on the header to see if you need to invert it according to your endiannes.

Also, you can save endiannes-specific files on your server and use them accordingly to the detected host endiannes.

Answer 4

The other answers seem a bit outdated to me, so here's a link to the latest spec:

http://www.khronos.org/registry/typedarray/specs/latest/#2.1

In particular:

The typed array view types operate with the endianness of the host computer.

The DataView type operates upon data with a specified endianness (big-endian or little-endian).

So if you want to read/write data in Big Endian (Network Byte Order), see: http://www.khronos.org/registry/typedarray/specs/latest/#DATAVIEW

// For multi-byte values, the optional littleEndian argument
// indicates whether a big-endian or little-endian value should be
// read. If false or undefined, a big-endian value is read.

Answer 5

Quick way to check endianness

/** @returns {Boolean} true if system is big endian */
function isBigEndian() {
    const array = new Uint8Array(4);
    const view = new Uint32Array(array.buffer);
    return !((view[0] = 1) & array[0]);
}

How it works:

• an array of 4 bytes is created;
• a 32-bit view wraps that array;
• view[0] = 1 sets the array to hold 32-bit value 1;
• now comes the important part: if system is big endian, that 1 is being hold by the rightmost byte (little comes last); if it is little endian, it is the leftmost byte that stores it (little comes first). So doing a bitwise AND with the leftmost byte returns false if the machine is big endian;
• the function finally converts it to a boolean by applying the ! operator to the result of the & operation, while also inverting it so that it returns true for big endian.

One nice tweak is to turn it into an IIFE, that way you can run the check only once and then cache it, then your application can check it as many times as it needs:

const isBigEndian = (() => {
    const array = new Uint8Array(4);
    const view = new Uint32Array(array.buffer);
    return !((view[0] = 1) & array[0]);
})();

// then in your application...
if (isBigEndian) {
    // do something
}

Answer 6

Yet Another QUICK WAY to CHECK Endianness:

Just adding my 2Cents here, but, my preferred method below is something I’ve found useful; especially when stored statically in a Singleton and made available across classes:

static isLittleEndian = (function(){
            var a8 = new Uint8Array(4);
            var a32 = new Uint32Array(a8.buffer)[0]=0xFFcc0011;
            return !(a8[0]===0xff);
        })();

If each 8Bits is not stored in the same order as the hex was input, then it’s using little endian. The result is then stored, and is useful for further reference. The reason the result is accurate, is because data is stored in the buffer, the same way as it is stored Natively on the device, according to the ECMA script spec.

The fact it calls only once, and then stores it, is very useful; especially with a million+ Itterations, all needing to know which endianness to use, including the most obvious one, rendering.

To illustrate this:

const isLittleEndian = (function(){
    console.log("isLittleEndian function called");
    var a8 = new Uint8Array(4);
    var a32 = new Uint32Array(a8.buffer)[0]=0xFFcc0011;
    return !(a8[0]===0xff);
})();

for(let i = 0; i!=5; i++){
  if(isLittleEndian){
    console.log("Little Endian");
  }else{
    console.log("Big Endian");
  }
}

It’s similar to the isBigEndian version already posted, just done a the other way round; which is in the spirit of EndianNess.

Answer 7

this should return true on little-endian and false on big-endian:

function runtimeIsLittleEndian(){
    return (new Uint8Array(new Uint16Array([1]).buffer)[0] === 1);
}

because little-endian sets [0] to 1 and [1] to 0, conversely big endian set [0] to 0 and [1] to 1... i think? don't actually have a big-endian system available to test on.