Gzip in frontend - equivalent to node-gzip

Question

Let's assume I have the following in NodeJS:

import {
  gzip
} from "node-gzip";

const samlToken = fs.readFileSync(
  "./localDevToken.txt",
  "utf8",
);

const bufferSamlToken = Buffer.from(
  samlToken.replace("\r\n", ""),
  "utf8",
);

const gZipToken = await gzip(bufferSamlToken);
localDevToken = gZipToken
  .toString("base64")
  .replace(/\+/g, "-")
  .replace(/\//g, "_")
  .replace(/=+$/g, "");

And I want to do the same in the frontend. How can I achieve it ?

This is what I've tried using the Pako library from https://github.com/nodeca/pako

function convertSamlToken(input) {
  var samlToken = input.replace("\r\n", "");
  samlToken = pako.gzip(samlToken, {
    to: 'string'
  });
  samlToken = btoa(samlToken)
    .replace(/\+/g, "-")
    .replace(/\//g, "_")
    .replace(/=+$/g, "");

  return samlToken;
}

But the output is different. What is wrong ?

Not sure it's the only issue, but pako.gzip returns an Uint8Array, calling `btoa` on it will first convert that Uint8Array to its string representation: a series of numbers between 0 and 255 separated by commas, and then turn that string into its b64 representation. That's not what you want at all, you want the actual bytes to be turned to their base64 representation. Have a look at https://stackoverflow.com/questions/12710001/how-to-convert-uint8-array-to-base64-encoded-string to do it properly — Kaiido, Feb 04 '21 at 07:52

score 1 · Accepted Answer · answered Feb 04 '21 at 21:51

You are using pako incorrectly, there is no {to: 'string'} option and so the output is a Uint8Array. Here is how your function should look:

function convertSamlToken(input) {
  const samlToken = input.replace("\r\n", "");
  const bytes = pako.gzip(samlToken);

  // Convert Uint8Array to base64 in a safe way
  // See https://stackoverflow.com/a/9458996/7448536
  let binary = "";
  for (let i = 0; i < bytes.byteLength; i++) {
    binary += String.fromCharCode(bytes[i]);
  }

  return btoa(binary)
    .replace(/\+/g, "-")
    .replace(/\//g, "_")
    .replace(/=+$/g, "");
}

I've tested this function on random UTF-8 data and it produces the exact same output except for the 10th byte since that is the OS ID. This is because node-gzip sets it to 0x0A/TOPS-20 and pako uses 0x03/Unix. If that is a problem then just add bytes[9] = 0x0a; after line 3.

Gzip in frontend - equivalent to node-gzip

1 Answers1