Go and Node.js base64 encoding, decoding problems

Question

I am base64 encoding a string in Go

Then decoding in javascript (I have tried 3 different methods)

I cannot get the javascript result to match the original Go string

Go (encode)

a := []byte {138,143,163,224,178,73,161,15,240,121,53,192,198,182,52,245}
fmt.Println("a", string(a), a, len(a))
b := base64.StdEncoding.EncodeToString(a)
fmt.Println("b", b, []byte(b), len([]byte(b)))

js (decode)

 const b = [105,111,43,106,52,76,74,74,111,81,47,119,101,84,88,65,120,114,89,48,57,81,61,61];
 let bString = aesjs.utils.utf8.fromBytes(b);
 console.log("b", bString, b, b.length);

 let a1String = atob(bString);
 let a2String = Base64.decode(bString);
 let a3String = Buffer.from(bString, 'base64').toString('utf8');
 let a1 = aesjs.utils.utf8.toBytes(a1String);
 let a2 = aesjs.utils.utf8.toBytes(a2String);
 let a3 = aesjs.utils.utf8.toBytes(a3String);
 console.log("a", a1, a1.length, a2, a2.length, a3, a3.length);

All 3 methods fail, i.e. a1 != a, a2 != a, a3 != a

I assumed base64 encoding/decoding would be simple

What am I missing? Thanks

EDIT: The js code had a 'typo' - but the problem remains, that ai != a, even a1 != a2 == a3

EDIT: If the original a is simple, then everything works. But when a contains non-printable chars, I cannot decode in js Example where it works: a := []byte {65, 65, 65, 65, 65, 65, 65, 65, 65, 65, 65, 65, 65, 65, 65, 65} Here it does not works: a := []byte {138, 65, 65, 65, 65, 65, 65, 65, 65, 65, 65, 65, 65, 65, 65, 65}

Your go code is correct. So the problem is on the JS side. – colm.anseo Apr 17 '19 at 21:56 — colm.anseo, Apr 17 '19 at 21:56

colm.anseo · Answer 1 · 2019-04-17T23:42:58.527

2

From: https://github.com/ricmoo/aes-js/blob/master/README.md

UTF8 should NOT be used to store arbitrary binary data as it is a string encoding format, not a binary encoding format

So don't used utf8 related functions to convert raw (non utf8) binary data.

edited Apr 17 '19 at 23:42

answered Apr 17 '19 at 22:19

colm.anseo

19,337
4
43
52

2

And what is the solution then? – imran Apr 17 '19 at 23:52

score 1 · Accepted Answer · answered Apr 18 '19 at 04:26

In Go, strings are UTF8:

in := "Good afternoon, or こんにちは"
buf := []byte(in)
str := base64.StdEncoding.EncodeToString(buf)
fmt.Println(str)

That prints the base64 encoded (ASCII safe) representation: R29vZCBhZnRlcm5vb24sIG9yIOOBk+OCk+OBq+OBoeOBrw==

JS strings are UTF-16. So you need to both decode from base64 and convert from utf8 to utf-16. You could use a library, or a helper function, for example:

function b64DecodeUnicode(str) {
    return decodeURIComponent(atob(str).split('').map(function(c) {
        return '%' + ('00' + c.charCodeAt(0).toString(16)).slice(-2);
    }).join(''));
}

var str = "R29vZCBhZnRlcm5vb24sIG9yIOOBk+OCk+OBq+OBoeOBrw==";
var result = b64DecodeUnicode(str);
"Good afternoon, or こんにちは"

The b64DecodeUnicode() function is copied from this answer, see also the link to MDN for detailed info.

This works for proper UTF8 strings, so I will mark it as correct. Unfortunately, it does not solve my problem, which was not in scope here though: My data that gets encoded is binary (result of an encryption), and I think here the comment of @colminator might apply — imran, Apr 18 '19 at 08:45

Go and Node.js base64 encoding, decoding problems

2 Answers2