Get a JSON file (that contains Hebrew values) from a remote server and parse it in Node.js

Question

I have a dynamic JSON file hosted on a remote server (acting as some kind of an API), and it also contains some Hebrew text in its values.

How can I save the response and parse it a a JSON object?

That's what I've got so far using Request (https://www.npmjs.org/package/request):

var options = {
    url: 'http://www.AWEBSITEHERE.com/file.json',
    method: 'GET'
};

function callback(error, response, body) {
    if (!error && response.statusCode == 200) {
        var info = JSON.parse(body);
        // ...
    }
}

request(options, callback);

And this code gives the following error when accessing:

SyntaxError: Unexpected token ∩┐╜
    at Object.parse (native)
    at Request.callback [as _callback] (C:\Sites\node\proj\routes\inde
x.js:21:29)
    at Request.self.callback (C:\Sites\node\proj\node_modules\request\
request.js:122:22)
    at Request.EventEmitter.emit (events.js:98:17)
    at Request.<anonymous> (C:\Sites\node\proj\node_modules\request\re
quest.js:1019:14)
    at Request.EventEmitter.emit (events.js:117:20)
    at IncomingMessage.<anonymous> (C:\Sites\node\proj\node_modules\re
quest\request.js:970:12)
    at IncomingMessage.EventEmitter.emit (events.js:117:20)
    at _stream_readable.js:920:16
    at process._tickCallback (node.js:415:13)

I think this error has something to do with the BOM characters that the server sends.

How can I solve this problem?

Thanks!

Can you provide a sample of the raw response? Characters that fall outside of the ASCII range are supposed to be represented by `\u####` in JSON - if they're not, the server's response JSON may be invalid. — g.d.d.c, Jul 10 '14 at 16:00
Sorry for being such as noob... How can I check the encoding of the response text and capture the raw response? — Ron, Jul 10 '14 at 17:21
@Ron, Check my answer here: http://stackoverflow.com/questions/34150253/converting-from-windows-1255-to-utf-8-in-node-js — IsraGab, Apr 03 '16 at 17:01

score 0 · Answer 1 · edited Apr 26 '22 at 16:20

You may want to try decoding the json response body:

request(options, function (error, response, body) {
    if (!error && response.statusCode == 200) {
        body = body.toString('utf-8');
        var info = JSON.parse(body);
    }
}

If that doesn't work, it could be because the body is gzipped so it must extracted before you will be able to decode:

var encoding = response.headers['content-encoding']
if(encoding && encoding.indexOf('gzip')>=0) {
    body = uncompress(body);
}

Get a JSON file (that contains Hebrew values) from a remote server and parse it in Node.js

1 Answers1