How can I force Node's request module to not encode the response?

Question

I'm using Node's request module.
The response I get is "gziped" or otherwise encoded.

How can I
1. Build the request to not encode the response?
2. Decode the response?
The data is coming from http://api.stackexchange.com.

var myRequest = require('request');
var zlib = require('zlib');

var stackRequest = require('request'); 
var apikey = '<MyKey>';
var fromdate = '1359417601';
var tagged = 'node.js';
stackRequest(
    { method: 'GET'
        , uri: 'http://api.stackexchange.com/2.1/questions?key=' + apikey + 
          '&site=stackoverflow&fromdate=' + fromdate + '&order=desc&' + 
          'sort=activity&tagged=' + tagged + '&filter=default'
}, function(err, response, body) { 
    console.log(response.body); // How can I decode this?
});

[this might have the answer you are looking for](http://stackoverflow.com/questions/8880741/node-js-easy-http-requests-with-gzip-deflate-compression) — adrusi, Jan 30 '13 at 05:24

loganfsmyth · Accepted Answer · 2013-01-30T06:14:42.430

The encoding has nothing to do with request. StackOverflow's API returns GZip encoded data always, as explained in the API documentation. You need to use Node's zlib module to unzip the contents. This is a simple example:

var zlib = require('zlib');

// Other code

, function(err, response, body) {
   zlip.gunzip(body, function(err, data){
     console.log(data);
  });
});

The main downside of this, which is bad, is that this forces the request module to process the entire response content into one Buffer as body. Instead, you should normally use Node's Stream system to send the data from the request directly through the unzipping library, so that you use less memory. You'll still need to join the parts together to parse the JSON, but it is still better.

var zlib = require('zlib');
var request = require('request');

var apikey = '<MyKey>';
var fromdate = '1359417601';
var tagged = 'node.js';
var compressedStream = request('http://api.stackexchange.com/2.1/questions?' +
    'key=' + apikey + '&site=stackoverflow&fromdate=' + fromdate +
    '&order=desc&sort=activity&tagged=' + tagged + '&filter=default');

var decompressedStream = compressedStream.pipe(zlib.createGunzip());

var chunks = [];
decompressedStream.on('data', function(chunk){
  chunks.push(chunk);
});
decompressedStream.on('end', function(){
    var body = Buffer.concat(chunks);

    var data = JSON.parse(body);

    // Do your thing
});

score 0 · Answer 2 · answered Jan 30 '13 at 08:15

0

First set accept: identity as a header. If stacked change doesn't send data as regular UTF8, then it's a bug on their end.

Secondly, you want to set the encoding as UTF8 so the response isn't a buffer.

answered Jan 30 '13 at 08:15

Jonathan Ong

19,927
17
79
118

How can I force Node's request module to not encode the response?

2 Answers2