4

I am trying to parse a JSON response using Node & Request. The JSON response comes from Apple, so I assume it is perfectly valid JSON. Additionally, if I use Post Master on Chrome to send the request, the raw response appears to be perfectly valid. I have even used burp suite to tap into the raw byte response - incase there is an erroneous byte in the response - again, all seems good.

However, the following code prints out complete garbage for every log statement (I have tried this on multiple machines and with different versions of Node & Request to no avail):

request(postOptions, function(error, response, body) {
    if (!error && response.statusCode == 200) {

            console.log(body);
            console.log(body.toString());
            console.log(JSON.parse(body));

    }               
});

Where postOptions is:

var postOptions = {
    url: '_THE_VALID_URL_',
    followAllRedirects: true,
    method: 'POST',
    body: '_THE_VALID_BODY_',
    headers: {

        'Connection': 'keep-alive',
        'Cache-Control': 'no-cache',
        'User-Agent': 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10_2) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/40.0.2214.115 Safari/537.36',
        'Accept': '*/*',
        'Accept-Encoding': 'gzip, deflate',
        'Accept-Language': 'en-US,en;q=0.8,pt;q=0.6',

    }
}

I have made sure all the parameters are the same as the ones I am using when I make the POST request using Post Master.

What gets printed looks like:

�R�n1�k���-���V�$Pr�Ib��_Ҕ���&i��# ��{Ι���Yp��ފh��~��eNKX�b��n����|�2H�D��eD�^y���J�B��,��fk�uDj��@���P�J��ɍ��pc-pO��uaQIc���>�^hd�0%�5�<������G��#�5���m��YV�{H�� �C����o�o�N�/8�b����虬�yV�8@�f%M��ϲb\�t�S���'����M�|��2� o_� �{ÙB�#����S��*<{_�q؀25ؘ��sSa�^� �Frp�qn��ɾVr S%��l�.f��7ڃ��?�0���ɖ�(��P��� �~�T��U ����

Can someone please help me out here, not sure what to try next! Thanks

Kevin B
  • 94,570
  • 16
  • 163
  • 180
user3703938
  • 221
  • 2
  • 10
  • gzipping maybe? http://stackoverflow.com/questions/8880741/node-js-easy-http-requests-with-gzip-deflate-compression guess you could just remove that header from the request you are sending so you get a non-gzipped response if that's the problem. – Kevin B May 06 '15 at 15:16
  • just removed the `gzip` part from `Accept-Encoding` but still garbage is being printed :( – user3703938 May 06 '15 at 15:23
  • just checked the `response.headers` and the `content-encoding` is `gzip` despite not accepting it. I will have a look further into whether the gzipping is causing the problem – user3703938 May 06 '15 at 15:27
  • Try removing the `Accept-Encoding` header altogether, or *only* putting "identity" in there. – Jason May 06 '15 at 15:28
  • Or, for that matter, try to run gunzip on it just to see what you get – Jason May 06 '15 at 15:28
  • Thanks @KevinB gzipping was the issue! I found out that by default the request module does not decompress gzipped responses. Including `gzip: true` in the `postOptions` ensured the response was decompressed and no more garbage! – user3703938 May 06 '15 at 15:31

1 Answers1

10

Thanks to @KevinB - gzipping was the issue.

By default the request module does not decompress gzipped responses. Including gzip: true in the postOptions ensured the response was decompressed and no more garbage!

user3703938
  • 221
  • 2
  • 10