0

I'm writing a javascript client using AES-128 to communicate with a server written in java. The server generates a password from a string using SHA1 hashing. The data is then encrypted for the server. This works fine for ASCII characters, but fails for higher unicode chars in javascript because the js strings are in UTF-16, so hashing the bytes of the string gives a different result. The test suite I'm using has a password that contains £. This is 0x00a3 in UTF-16 and 0xC2 0xA3 in UTF-8. I can't figure out how to convert the UTF-16 unicode code points into a uint8array of UTF-8. I've tried with TextEncoder/TextDecoder unsuccessfully.

The following code shows the problem. Using java, C# and perl, the expected base64 encoded hash, truncated to 16 bytes for AES-128 key, comes out as +guOwqu6/gVmQzArLrzGQg==. Running the javascript below gives me 1nNTS1vHThwvV7lUI0EB9g==.

const crypto = require('crypto');

function getSecretKey(key) {
    var hasher = crypto.createHash('sha1');
    const data = hasher.update(Buffer.from(key, 'utf16le'));
    return data.digest().slice(0, 16).toString('base64');
}

console.log(getSecretKey("abcd£"));

Here's a perl program to do the same thing, almost. It doesn't truncate to 16 bytes, but the output should match until the last few character.

use Digest::SHA1  qw(sha1 sha1_hex sha1_base64);
$digest = sha1_base64("abcd£");
print $digest;
print "\n";

There must be a way to create a Buffer or uint8Buffer of UTF-8 chars from a javascript string of UTF-16 codepoints.

jsparkes
  • 161
  • 1
  • 11
  • Even though the in-memory representation in javascript might resemble something UTF-16ish, normally when you use strings in hash functions they are treated as UTF-8 strings. Using UTF-8 instead of UTF16LE should fix it(?) – Evert Nov 01 '20 at 04:55
  • I found a solution in [this stackoverflow question](https://stackoverflow.com/questions/18729405/how-to-convert-utf8-string-to-byte-array) – jsparkes Nov 03 '20 at 06:55
  • Does this answer your question? [How to convert UTF8 string to byte array?](https://stackoverflow.com/questions/18729405/how-to-convert-utf8-string-to-byte-array) – Joe Nov 22 '20 at 10:35

0 Answers0