Client JavaScript Problem in unzipped Gzip data form webSocket

Question

I have a service that sends me data through WebSocket however the data is gzipped. when I try to ungziped the Blob data I have a problem. I used packages fflate gzip-js pako and zlib all with different error but the funny thing when I write the code for node js Server with package node-gzip which underhood used the zlib package mention above everything is fine and the data will be ungzped.

Client.js (Problem is unGzip data)

let ws = new WebSocket('wss://api.huobi.pro/ws');

var zlib = require('zlib');;

let zip = {
    gzip: function gzip(input, options) {
        return new Promise(function (resolve, reject) {
            zlib.gzip(input, options, function (error, result) {
                if (!error) resolve(result); else reject(Error(error));
            });
        });
    },
    ungzip: function ungzip(input, options) {
        return new Promise(function (resolve, reject) {
            zlib.gunzip(input, options, function (error, result) {
                if (!error) resolve(result); else reject(Error(error));
            });
        });
    }
};

ws.onopen = ( function () {
    ws.send(`{
                "sub": "market.ethbtc.kline.1min",
                "id": "id1"
            }`);
});

ws.onmessage = async function (event) {
    console.log(await zip.ungzip(event.data));
};

ERROR : Uncaught (in promise) Error: TypeError: Invalid non-string/buffer chunk

Node.js Working

const {ungzip} = require('node-gzip');

ws = new WebSocket('wss://api.huobi.pro/ws');
    
    ws.on('open', function () {
            ws.send('{
                "sub": "market.ethbtc.kline.1min",
                "id": "id1"
            }');
        });
    
     ws.on('message', function (data) {
            ungzip(data).then((r) => {
                let data = JSON.parse(r.toString());
                console.log(data); 
            }).catch(console.error)
        });

In the browser the event.data is Blob data when I used the text method on them there are some jibberish code and most of the library and packages above have said the data header is unknown

but why same data point with the same structure one working on node.js another not working in the browser

Please help me unzipped the data

sample response data after unGziped in Node.js code above I want the same in my Client.js

{
  "ch": "market.ethbtc.kline.1min",
  "ts": 1489474082831, //system update time
  "tick": {
    "id": 1489464480,
    "amount": 0.0,
    "count": 0,
    "open": 7962.62,
    "close": 7962.62,
    "low": 7962.62,
    "high": 7962.62,
    "vol": 0.0
  }
}

score 3 · Accepted Answer · answered Dec 28 '20 at 04:41

If you console.log the value of event.data, you'll see it's actually a Blob. You can't use this directly, instead you must convert to Uint8Array.

I also noticed that endpoint returns gzipped JSON, so I'll parse that too.

My solution:

import { gunzip, strFromU8 } from 'fflate'; 
// If you can't use ES Modules:
// const { gunzip, strFromU8 } = require('fflate');
const ws = new WebSocket('wss://api.huobi.pro/ws');
ws.onopen = function() {
  ws.send(JSON.stringify({
    sub: 'market.ethbtc.kline.1min',
    id: 'id1'
  }));
}
ws.onmessage = function(event) {
  const fr = new FileReader();
  fr.onload = function() {
    gunzip(
      new Uint8Array(fr.result),
      function(err, raw) {
        if (err) {
          console.error(err);
          return;
        }
        const data = JSON.parse(strFromU8(raw));
        // Use the data variable however you wish
      }
    );
  }
  fr.readAsArrayBuffer(event.data);
}

thank u, I think the browser put gzip data to Blob Object the same code work in Express application Just fine that why I couldn't do anything I never work with Blob Object and try reading them in the hope maybe the data is unGziped the but no the UnGzip data was the Data thanks again :) — Milad, Dec 28 '20 at 11:58

Martin Zeitler · Answer 2 · 2020-12-27T23:06:23.723

0

There is an obvious difference in between the two examples, no matter the pattern used.
In case this won't work (because it tries to access instantly, without waiting for the promise):

ws.onmessage = function (event) {
    console.log(gzip.unzip(event.data));
}

Just access it the same way as in the working version:

ws.onmessage = function (data) {
    gzip.unzip(data)
    .then((result) => {
        let data = JSON.parse(result.toString());
        console.log(data); 
    })
    .catch(console.error)
}

gzip.unzip and ungzip might both return a promise.

edited Dec 27 '20 at 23:06

answered Dec 27 '20 at 23:01

Martin Zeitler

1
19
155
216

Sorry I update my Client.js I have done that just want to be short code. still have error – Milad Dec 27 '20 at 23:08
I honestly (nor does NodeJS) care what you want; you'll have to use what it returns. You simply can't just use an asynchronous method in a synchronous manner; this will never work out. – Martin Zeitler Dec 27 '20 at 23:10
the data is Node.js can be used but in Client (Browser side) is unusable – Milad Dec 27 '20 at 23:12
@Milad Please had a look here: https://nodejs.dev/learn/understanding-javascript-promises ...which might explain in more detail, what I'm trying to tell. A `Promise` isn't a result, but it promises you to deliver a result in the future ...that's why you'd have to wait with `.then()`. I wouldn't even be that sure, that the gzip wouldn't be transparently unzipped. – Martin Zeitler Dec 29 '20 at 07:14
thanks, @martin-zeitler I know about Promise the problem I have was about Reading Blob data the browser put gzip data inside Blob Object that why I couldn't unzip data as 101arrowz show me how to open Blob data and then unzip that data thanks again for your effort to help me :) – Milad Dec 29 '20 at 16:53

Client JavaScript Problem in unzipped Gzip data form webSocket

2 Answers2

Linked