Json Decode and Parse Error node.js

Question

Hi I'm trying to do this

const request = require('request');
const zlib = require('zlib');
const opts = {
    uri: 'http://data.githubarchive.org/2015-01-01-15.json.gz',
    encoding: null,
};

request.get(opts, function(error, response, body) {
    if (!error) {
        zlib.gunzip(body, function(err, decoded) {
            if (err) {
                console.log(err)
            } else {
                var json_string = decoded.toString('utf-8').replace(/(\r\n|\n|\r)/gm, " ").trim();
                var json = JSON.parse(json_string);
                console.log("SJON", typeof json)
            }
        });
    }
});

I'm following the below steps:

fetching data from url
unzip that using zlib.gunzip
converting that decoded data to string
replacing all newline and beak statements
I'm trying to parse that string which throws error

I'm getting error while parsing data using JSON.parse, this is public dataset of github. I don't know where I'm going wrong, can any one help.

What kind of error are you getting? Have you tried printing out `json_string` before decoding it? Is it a valid JSON? — Eduard Malakhov, Apr 08 '17 at 07:01
yes i printed thus string it is correct. i am getting parse error unexpected token at position 571. even i saved that string to file — Rayees, Apr 08 '17 at 07:04
Use a debugger (like `node-inspector`) to examine the contents of `json_string` before parsing. (Or in this case, `console.log` may be sufficient; normally a debugger is better, you get much better insight into what's happening.) — T.J. Crowder, Apr 08 '17 at 07:07
*"yes i printed thus string it is correct"* Almost certainly not. The `JSON.parse` in Node isn't broken. Show the string. — T.J. Crowder, Apr 08 '17 at 07:07

score 3 · Answer 1 · answered Apr 08 '17 at 07:07

That file contains one JSON object per line. Just removing the newlines won't yield a valid single JSON object. Instead, you can split the input data on newlines, and parse each line separately:

zlib.gunzip(body, function(err, decoded) {
  if (err) {
    console.log(err);
  } else {
    let array = decoded.toString().trim().split(/\r?\n/).map(line => JSON.parse(line));
    ...
  }
});

Json Decode and Parse Error node.js

1 Answers1