Node.js buffer to typed array

Question

In node.js I have a Buffer (It was stored as a blob in mysql and retrieved using sequelize) I know that this Buffer is an array of 16 bit integers. In the past I've parsed the code with a for loop.

    var spectrum_buffer = spectrums[idx]["spectrum"];//this is a buffer
    var parsed_spectrum = [];
    for (var i = 0; i < spectrum_buffer.length / 2; i++) {
        parsed_spectrum[i] = spectrum_buffer.readInt16BE(i * 2);
    }

I've read that readInt16BE is slow and that there are typedarrays for arraybuffers now. (which are different than Buffers). Is there a better way of creating an array of ints from this buffer.

Update 1

Based on the feedback I did the following

var arr = new Int16Array(spectrum.buffer)

Which gives me the appropriate type however the bytes are getting swapped. The spectrum buffer is stored in big endian.

< Buffer e1 d7 e0 b9 e3 52 e2 d5 e2 ed e2 92 e2 d6 e2 97 e3 04 e1 95 e1 e2 > e1 d8 e3 14 e2 fd e1 ed e2 d3 e3 09 e1 9f e2 14 e2 f2 e2 54 e2 1f e2 54 > e2 06 e2 8a ... >

The first three numbers are coming across as -10271, -17952, 21219

However they should not vary that much and all three should be negative.

The first number should be -7721 (twos complement always confuses me)

So does Int16Array on node 6 assume big endian or little endian and how do I handle this.

Note: https://nodejs.org/api/buffer.html#buffer_buffers_and_typedarray Sure looks like it's possible, if you create the typed array *before* filling the buffer. — T.J. Crowder, Jun 13 '16 at 16:33
Just so you know, I updated my answer. There's an even lower level interface you can use if you need to read the data with a certain endianess. — Mike Cluck, Jun 13 '16 at 20:32

Mike Cluck · Accepted Answer · 2016-06-13T20:32:16.990

4

Sure. Node.js buffers are a special instance of Uint8Array.

So if you wanted to create an instance of a Int16Array you could create a copy of your buffer:

var int16Arr = new Int16Array(spectrum_buffer);

or create a new array which references the same underlying buffer, which means you don't have to copy all of the data:

var int16Arr = new Int16Array(spectrum_buffer.buffer);

Update:

Typed arrays default to using the native byte order. If you need to specify the byte order, you can use a lower-level interface: DataView.

A DataView is a wrapper around a Buffer which gives you full control over how that data is accessed. In your case, you would want to use the getInt16 method and set the endianess flag as needed for your data:

var littleEndian = true; // or false, depends on your needs
var dataView = new DataView(spectrum.buffer);
dataView.getInt16(offset, littleEndian);

edited Jun 13 '16 at 20:32

answered Jun 13 '16 at 16:35

Mike Cluck

31,869
13
80
91

4

Both `new Int16Array` calls above will create 1 value per byte, not recreate the typed array from the byte data. – Tomáš Zato Jul 17 '18 at 12:02
@TomášZato-ReinstateMonica If you want to recreate a typed array from the byte data, see [this question](https://stackoverflow.com/q/30604777/2441655) and [the accepted answer](https://stackoverflow.com/a/31483629/2441655) for background, then read the ["best solution" link](https://stackoverflow.com/a/31394257/2441655) for how to do this. – Venryx Dec 10 '20 at 11:22

score 0 · Answer 2 · answered Jun 13 '16 at 19:37

Try this:

//this is a buffer
var spectrum_buffer = spectrums[idx]["spectrum"];

// these two views share same memory
var int16view = new Int16Array(spectrum_buffer); 
var uint8view = new Uint8Array(spectrum_buffer);

var parsed_spectrum = [];

for (var i = 0; i < int16view.length; i++) {
    // swap byte order
    [uint8view[i*2], uint8view[i*2+1]] = [uint8view[i*2+1], uint8view[i*2]]; // ES6 swap
    // read swapped bytes as Int16
    parsed_spectrum[i] = int16view[i];
}

Node.js buffer to typed array

2 Answers2