Converting javascript Integer to byte array and back

Question

function intFromBytes( x ){
    var val = 0;
    for (var i = 0; i < x.length; ++i) {        
        val += x[i];        
        if (i < x.length-1) {
            val = val << 8;
        }
    }
    return val;
}

function getInt64Bytes( x ){
    var bytes = [];
    var i = 8;
    do {
    bytes[--i] = x & (255);
    x = x>>8;
    } while ( i )
    return bytes;
}

I am trying to convert a javascript number to a byte array and then back to a number. However the above functions produce incorrect output with a very large number.

var array = getInt64Bytes(23423423);    
var value = intFromBytes(array);

console.log(value); //Prints 23423423 - correct

var array = getInt64Bytes(45035996273704);  
var value = intFromBytes(array);

console.log(value); //Prints -1030792152 - incorrect

It is my understanding that javascript floats are 53 bits so it shouldn't be overflowing? alert(Math.pow(2,53)) works fine.

Bit shifts are always performed on signed, 32-bits integers. — Crozin, Dec 12 '11 at 23:18
Since this question ranks high in google, it's probably worth pointing out that there are now array buffers: https://stackoverflow.com/questions/15761790/convert-a-32bit-integer-into-4-bytes-of-data-in-javascript — Melvin Sovereign, Feb 07 '20 at 15:56

score 58 · Answer 1 · answered Oct 18 '12 at 23:19

58

Using the hint provided by Susanoh13, here are the two functions that allow conversion of number from/to ByteArray:

longToByteArray = function(/*long*/long) {
    // we want to represent the input as a 8-bytes array
    var byteArray = [0, 0, 0, 0, 0, 0, 0, 0];

    for ( var index = 0; index < byteArray.length; index ++ ) {
        var byte = long & 0xff;
        byteArray [ index ] = byte;
        long = (long - byte) / 256 ;
    }

    return byteArray;
};

byteArrayToLong = function(/*byte[]*/byteArray) {
    var value = 0;
    for ( var i = byteArray.length - 1; i >= 0; i--) {
        value = (value * 256) + byteArray[i];
    }

    return value;
};

answered Oct 18 '12 at 23:19

Younes

1,635
1
19
33

1

Sometimes in byteArrayToLong() byteArray[i] is treated as a string (browser dependent) and the resultant value is calculated wrongly. I have solved by *1: value = (value * 256) + byteArray[i] * 1; – DrArt Jan 23 '14 at 16:18
1

You should specify whether this considers the bytes as containing signed or unsigned integers. – Drew Noakes Oct 18 '17 at 20:44
2

This doesn't work for negative numbers. Instead, large positive numbers are decoded. – ygoe Jun 19 '18 at 15:05
4

DON'T USE THIS CODE. This code would only be valid if Javascript numbers were 64 bit unsigned integers. However they are 64 bit floating point numbers, which only store up to 52 bits as an exact integer. This means that while this code works for many bytes (ex, [0,1,2,3,4,5,6,7]), it will fail when all of the bits are actually used. Some failing cases are(where converting to a number and back again give a different result): ([255,255,255,255,255,255,255,255] => [0, 0, 0, 0, 0, 0, 0, 255]), ([1,2,3,4,5,6,7,8] => [0, 2, 3, 4, 5, 6, 7, 8]), ([1,0,0,0,0,0,0,1] => [0, 0, 0, 0, 0, 0, 0, 1]) – yeerk Sep 12 '20 at 17:38

score 17 · Accepted Answer · answered Dec 12 '11 at 23:24

17

In JavaScript bit shifts (>>, <<) are always performed on signed, 32-bits integers. This leads to range overflow for large numbers.

answered Dec 12 '11 at 23:24

Crozin

43,890
13
88
135

1

But if long (64 bit signed integer) is needed, what is the solution? – Timo Kähkönen Nov 21 '13 at 13:43
1

JS doesn't support 64 bits-long integers. See this: http://stackoverflow.com/questions/9643626/javascript-cant-handle-64-bit-integers-can-it and http://stackoverflow.com/questions/5353388/javascript-parsing-int64 – Crozin Nov 21 '13 at 13:47
@TimoKähkönen `x< – Jun 04 '17 at 01:40
JavaScript (ESNext) supports 64-bit numbers now. Either signed and unsigned. – Derk Jan Speelman Feb 25 '20 at 12:55
@DerkJanSpeelman any link? unless you're referring to `BigInt`, but this isn't what I'd call a 64-bit number – timotheecour Dec 29 '20 at 22:04
@timotheecour https://tierion.github.io/boltwall/interfaces/_node_modules_typescript_lib_lib_esnext_bigint_d_.bigint64array.html – Derk Jan Speelman Jan 05 '21 at 17:45

Kamil Kiełczewski · Answer 3 · 2019-11-21T21:25:50.223

8

try (** is power operator, << and >>> are bit-shift operators) - intFromBytes works only for arrays generated from positive integers

function getInt64Bytes(x) {
  let y= Math.floor(x/2**32);
  return [y,(y<<8),(y<<16),(y<<24), x,(x<<8),(x<<16),(x<<24)].map(z=> z>>>24)
}

function intFromBytes(byteArr) {
    return byteArr.reduce((a,c,i)=> a+c*2**(56-i*8),0)
}

function getInt64Bytes(x) {
  let y= Math.floor(x/2**32);
  return [y,(y<<8),(y<<16),(y<<24), x,(x<<8),(x<<16),(x<<24)].map(z=> z>>>24)
}

function intFromBytes(byteArr) {
    return byteArr.reduce((a,c,i)=> a+c*2**(56-i*8),0)
}


// TEST

let n = 40*2**40 + 245*2**32 + 194*2**24 + 143*2**16 + 92*2**8 + 40;
let b = getInt64Bytes(n);
let i = intFromBytes(b);

console.log(`number      : ${n}`);
console.log(`int to bytes: [${b}]`);
console.log(`bytes to int: ${i}`);

edited Nov 21 '19 at 21:25

answered May 09 '19 at 14:15

Kamil Kiełczewski

85,173
29
368
345

The value for `y` in `getInt64Bytes` needs to be wrapped in `Math.floor` or else it breaks for negative integers. For example, `getInt64Bytes(-Number.MAX_SAFE_INTEGER)` returns `[255, 224, 0, 1, 0, 0, 0, 1]` when it should return `[255, 224, 0, 0, 0, 0, 0, 1]`. – snickle Nov 21 '19 at 21:12
@snickle this procedure works only for positive integers - the Math.floor not fix the problem in intFromBytes (e.g. when we put minus in let n= -... in snippet) - currently I don't have time to fix this problem - but fell free create your answer which develop my answer and fix this problem. – Kamil Kiełczewski Nov 21 '19 at 21:24

score 5 · Answer 4 · answered Aug 21 '22 at 15:38

In 2022, you should use Buffer to deal with bytes, and BigInt to deal with big integers.

So your code may look like this:

import { Buffer } from "node:buffer";

/**
 * @param {BigInt} x
 * @returns {Buffer}
 */
function getInt64Bytes(x) {
  const bytes = Buffer.alloc(8);
  bytes.writeBigInt64LE(x);
  return bytes;
}

/**
 * @param {Buffer} x 
 * @returns {BigInt}
 */
function intFromBytes(x) {
  return x.readBigInt64LE();
}

var array = getInt64Bytes(BigInt(23423423));
var value = intFromBytes(array);

console.log(value); // 23423423n

var array = getInt64Bytes(BigInt(45035996273704));
var value = intFromBytes(array);

console.log(value); // 45035996273704n

score 4 · Answer 5 · answered May 09 '17 at 03:08

4

Brainfuck-style Lodash version. Just 4 lulz! don't use it!

const uintToArray = (uint, size) => _.chunk(_.padStart(uint, size*2,  0).split(''), 2).map((a)=>parseInt(a[0]+a[1]))

answered May 09 '17 at 03:08

SET001

11,480
6
52
76

score 3 · Answer 6 · answered Jul 19 '12 at 15:42

3

Doing a bit shift is the same as multiplying by 2^(# of bits+1), so instead of shifting the bits val = val<<8, you can just do val = val*256. See if that works.

answered Jul 19 '12 at 15:42

StoicJester

364
2
12

score 1 · Answer 7 · answered Mar 03 '18 at 17:53

<html>
<head>
    <meta charset="utf-8">
    <title>Uint32_To_Byte_Array</title>
    <script>
    function body_Add(Msg)
    {
        document.body.innerHTML = document.body.innerHTML + Msg;
    }
    class Byte 
    {
        constructor(Value) 
        {
            this.Number = new Uint8Array(1);
            this.Number[0] = Value;
        }
        get Get() 
        {
            return this.Number[0];
        }
        set Set(newValue) 
        {
            this.Number[0] = newValue;
        }
    };
    class Uint32
    {
        constructor(Value) 
        {
            this.Number = new Uint32Array(1);
            this.Number[0] = Value;
        }
        get Get() 
        {
            return this.Number[0];
        }
        set Set(newValue) 
        {
            this.Number[0] = newValue;
        }
    };
    var Conversion =
    {
        Uint32_To_Byte_Array: function(Source_Num)
        {
            var Uint32_Num = new Uint32(Source_Num);
            var Byte_Num = new Byte(0);
            var Byte_Arr = new Uint8Array(4);
            for (var i = 0; i < 4; i++)
            {
                if (Source_Num > 255)
                {
                    Uint32_Num.Set = Source_Num / 256;
                    Byte_Num.Set = Source_Num - Uint32_Num.Get * 256;
                }
                else
                {
                    Byte_Num.Set = Uint32_Num.Get;
                    Uint32_Num.Set = 0;
                }
                Byte_Arr[i] = Byte_Num.Get;
                Source_Num = Uint32_Num.Get;
            }
            return(Byte_Arr);
        },
        Byte_Array_To_Uint32: function(Source_Byte_Array, Start_Position)
        {
            var Uint32_Num = new Uint32(0);
            var Multiplier = 1;
            for (let i = 0; i < 4; i++)
            {
                Uint32_Num.Set = Uint32_Num.Get + Source_Byte_Array[Start_Position + i] * Multiplier;
                Multiplier = Multiplier * 256;
            }
            return (Uint32_Num.Get);
        }
    };
    function Load_Page()
    {
        var Numbers = [0,1,257,4294967295];
        Numbers.forEach(Convert);
        function Convert(Item, Index)
        {
            var Uint32_Number = Item;
            var Byte_Array = Conversion.Uint32_To_Byte_Array(Uint32_Number);
            var Uint32_Number_Restored = Conversion.Byte_Array_To_Uint32(Byte_Array, 0);
            body_Add("Conversion: Source number: " + Uint32_Number.toString() + ", Byte array: " + Byte_Array.toString() + ", Restored number: " + Uint32_Number_Restored.toString() + "<br>");
        };
    };
    </script>
</head>
<body onload="Load_Page()"> 
</body>

Result:Conversion: Source number: 0, Byte array: 0,0,0,0, Restored number: 0; Conversion: Source number: 1, Byte array: 1,0,0,0, Restored number: 1; Conversion: Source number: 257, Byte array: 1,1,0,0, Restored number: 257; Conversion: Source number: 4294967295, Byte array: 255,255,255,255, Restored number: 4294967295 — Valery Rode, Mar 03 '18 at 17:54
This is exactly what I need! Just changing a bit :) Thanks!! :) — Jure, Jun 18 '21 at 09:24

score 1 · Answer 8 · answered May 20 '20 at 21:37

If you happen to be on Node.js, Buffer is the correct way to handle any byte array/stream in Javascript/Typescript:

https://nodejs.org/api/buffer.html

Although the docs are more comprehensive, Stack Overflow recommends code snippets here in case that link 404s, so here are a couple of the most important code examples in that doc:

// Creates a Buffer containing the UTF-8-encoded bytes for the string 'tést':
// [0x74, 0xc3, 0xa9, 0x73, 0x74] (in hexadecimal notation)
// [116, 195, 169, 115, 116] (in decimal notation)
const buf6 = Buffer.from('tést');

// Creates a Buffer containing the bytes [1, 2, 3].
const buf4 = Buffer.from([1, 2, 3]);

Yes, to use buffer is the correct way to handle any byte array when is already a buffer or byte array. But the main question is: How do you convert an INTEGER to a byte array using Buffer? Buffer.from(INTEGER) is not allowed. TypeError [ERR_INVALID_ARG_TYPE]: The "value" argument must not be of type number. Received type number at Function.from (buffer.js:215:11) — Alberto Torre, May 31 '20 at 08:31

Shashikant Soni · Answer 9 · 2021-08-24T08:00:47.057

1

Just so that someone coming here after node version 16.5 knows that the buffer API from node.js now provides a way to write/read various sizes of numbers to byte array and vice-versa using the

writeUIntXXX

apis. These cover integers, long, double, float, and in Big and Small Endian formats. So I guess we don't need to spin off our own solutions anymore.

edited Aug 24 '21 at 08:00

answered Aug 23 '21 at 11:06

Shashikant Soni

191
9

score 0 · Answer 10 · answered Jun 24 '21 at 06:03

This will work,

    let buf;

    if (num < 128)
    {
        buf = Buffer.from([num]);
    }
    else if (num < 256)
    {
        buf = Buffer.from([129, num]);
    }
    else if (num < 65536)
    {
        buf = Buffer.from([130, 256, num % 256]);
    }
    else if (num < 65536)
    {
        buf = Buffer.from([130, num / 256, num % 256]);
    }
    else if (num < 16777216)
    {
        buf = Buffer.from([131, num / 65536, num / 256, num % 256]);
    }


    console.log(buf.toString('hex'));

Could you please explain why or add some reference about the magic numbers 129,130,131? Also, there are two branches of `num<65536` which I believe is a typo. — Pablo LION, Oct 04 '22 at 14:12

metw · Answer 11 · 2022-11-05T20:14:25.140

Modern solution for modern browsers

// Number or BigInt to Uint8Array
NumberToUint8Array = number => {
    var array = [], bigint = BigInt(number)
    for (let i = 0; i < Math.ceil(Math.floor(Math.log2(new Number(number)) + 1) / 8); i++)
        array.unshift(new Number((bigint >> BigInt(8 * i)) & 255n))
    return new Uint8Array(array)
}

// Uint8Array to BigInt
Uint8ArrayToBigInt = Uint8Array => [...Uint8Array].reverse().reduce((prev, curr, index) => BigInt(prev) | (BigInt(curr) << BigInt(index * 8)))

// example
NumberToUint8Array(12312321312312309876543234567890n)
// returns: [155, 103, 65, 124, 45, 109, 221, 194, 90, 249, 152, 238, 210]
Uint8ArrayToBigInt([155, 103, 65, 124, 45, 109, 221, 194, 90, 249, 152, 238, 210])
// returns: 12312321312312309876543234567890n

Converting javascript Integer to byte array and back

11 Answers11

Linked

Related