Request package: response.body is not JSON

Question

I am using nodejs request module to fetch data from stackexchange api. However, I do not understand the body of the response

��V�n�6������$_��Ң�C��MQc�Ԓ���wHɉs�.l��s�>��㆕ڋ�
<C2��ݓ-�T��x]OȄ���T���Y�Z��2Cs�{ד;��<Ū�å��Ѭ�՚�O�x����i��+��x:��0�d$��ʬ1(��z]�R2�[�d�xnL��0��� %����T�a���f��hM�wV��Z�߭���($�z   �UA�+AJ�w�P;'�BY���L�6�n䖩��;����֏��X?��"B�'�Q5�z[�v+*ka0�(B݉�ޠ�i�)�1D�D���o�ٯ�&��d:FgPNsA}E�`:�
1H?j���Fy4O�h��;'�O����#2�H�=`Z)|Rq��J�o+�_�����I��~�d�[TI��pSظ�}��y�H8����-4
    <p���@�X�{�Y�d�4�jq��Z;����K�����}��:pu���NGt��ԟDӉ%l��M��8�&4������#d*� �Eʔbj�-, �u��
IpE��!Y�� `;�5��yX-�b)N&S)0�6�-���f��Q��8��f"i�+m�6.�M�sr�B�ST�x��y�6���l�3����
        <a[48�Mp@��(���b�'���RA�D�m���n\4.8�.��.���v5jjU���Y��25S�-�]���z�T��'�:�޲�d&�I[��7pv�:��Ф(�3���$g$�1��׮z`�0��=�+���5px��x����r��u�-�������V�}ڼ����,8L��o�%�ږ��wc�#mM��&v�N|���Z�q�pZ?J������K[���Rd��BfX/�O�#ֿ���˷
��3T�mGa�`۴�7�ƕ"[���T�}�����v_�T�@��A����g}����ӌN�]��K�,$΁gJ<��z?���ђ
k��Q�a  ����$V66�g�? �H�����1��5�c�Xi���0[i��܍�
�̞Ϝl�*��8d�����C����R0�i3��dfI�b�k]��^he�QX3�Ҏ ;�5/���X�r(��7Z�A.���tR�9D*�F�ű���V�w�o.�ɪt))4�_ҐUI<��ӻb%���'�d|��3���

I want the body to be in JSON format.

I tried using JSON.parse(body) but it throws this error

undefined:1 � ^

SyntaxError: Unexpected token in JSON at position 0

Here is the code on my nodeJS app

var questionId = req.params.id
request.get('https://api.stackexchange.com/2.2/questions/'+questionId+'?order=desc&sort=activity&site=stackoverflow&filter=!9Z(-wwYGT',
    {json:true}, (error, response, body) => {
        if(error) res.status(500).json(error)
        else{
            res.status(200).send(body)
        }
    })

This is all of your code? I bet you're getting gzip or deflate transfer encoding. — Brad, Oct 23 '19 at 23:08
@Brad yup, that is all my code. How do I know if I'm getting gzip? — x7R5fQ, Oct 23 '19 at 23:10
It's only the body of the response that has these weird characters! the rest of the response object is normal JSON. — x7R5fQ, Oct 23 '19 at 23:14
@Brad `https://api.stackexchange.com/2.2/questions/'+questionId+'?order=desc&sort=activity&site=stackoverflow&filter=!9Z(-wwYGT` — x7R5fQ, Oct 23 '19 at 23:24
I can read your code, but what I can't do is mind-read the question ID. :-) — Brad, Oct 23 '19 at 23:24
@Brad something like `https://api.stackexchange.com/2.2/questions/53684484?order=desc&sort=activity&site=stackoverflow` — x7R5fQ, Oct 23 '19 at 23:25
@Brad I think as you said, it has something to do with gzip according to this thread: https://stackoverflow.com/questions/2888647/json-url-from-stackexchange-api-returning-jibberish — x7R5fQ, Oct 23 '19 at 23:32

score 2 · Accepted Answer · answered Oct 23 '19 at 23:33

Stack Exchange is requiring gzip. The request module isn't requesting it by default, so it assumes that it doesn't need to decode it.

Enable gzip on the request:

const request = require('request');

request.get(
  {
    url: 'https://api.stackexchange.com/2.2/questions/53684484?order=desc&sort=activity&site=stackoverflow&filter=!9Z(-wwYGT',
    json:true,
    gzip: true
  }, (error, response, body) => {
    console.log({error, response, body});

  }
);

https://repl.it/repls/AppropriateSarcasticFilesize

score 1 · Answer 2 · answered Oct 23 '19 at 23:25

1

Assuming request is the npm request module, the get method allows only two parameters (URL or options with URL, and callback) while you are passing URL and options separately.

For example:

request.get({
  uri: 'https://api.stackexchange.com/2.2/questions/'+questionId+'?order=desc&sort=activity&site=stackoverflow&filter=!9Z(-wwYGT',
  json: true,
},(error, response, body) => {
  // ... callback here
});

answered Oct 23 '19 at 23:25

Joaquín Gatica

56
3

this does not work. here is online demonstration: https://repl.it/@programom/fetchStackExchange – x7R5fQ Oct 23 '19 at 23:29

Request package: response.body is not JSON

2 Answers2