2

I have a large JSON file of 121MB. I want to read its structure and the data. I am unable to read it with notepad++ and json editor plugin of chrome. I have tried reading it with java. simple json reading is also giving exception:

Exception in thread "main" java.lang.OutOfMemoryError: GC overhead limit exceeded

If i use jackson streaming API. How do I read the structure and data completely? I do not know its json structure keys and values. I used code like this:

import java.io.File;
import java.io.IOException;

import com.fasterxml.jackson.core.JsonFactory;
import com.fasterxml.jackson.core.JsonGenerationException;
import com.fasterxml.jackson.core.JsonParser;
import com.fasterxml.jackson.core.JsonToken;


public class JacksonStreamExample {
    public static void main(String[] args) {

        try {

            JsonFactory jfactory = new JsonFactory();

            /*** read from file ***/
            JsonParser jParser = jfactory.createParser(new File(
                    "E:\\NUST\\Semester 3\\region_descriptions.json"));
            while (jParser.nextToken() != JsonToken.END_OBJECT) {

            System.out.println(jParser.getCurrentName());
            jParser.nextToken();
            System.out.println(jParser.getText());
            }
            jParser.close();

        } catch (JsonGenerationException e) {

            e.printStackTrace();

        }catch (IOException e) {

            e.printStackTrace();

        }

    }

}

Is there any way to read it with any text or JSON viewer? or with a java code? If java code, how to read the JSON structure?

Toto
  • 89,455
  • 62
  • 89
  • 125
Imran Khurram
  • 377
  • 8
  • 18
  • possible duplicate of: https://stackoverflow.com/questions/159521/text-editor-to-open-big-giant-huge-large-text-files and https://stackoverflow.com/questions/18208524/how-do-i-read-a-text-file-of-about-2-gb – DevDio Nov 26 '17 at 10:23
  • If you know the structure you could use "jq" to filter out what you need and reduce the size. – Mathias Begert Nov 26 '17 at 10:55

2 Answers2

3

If you just want to preview the general structure and data in the file (and don't necessarily need to load the entire file in memory), you can use Cluria (http://www.cluria.com). It will allow for loading very large JSON files very quickly if you just want a preview of what's in the file.

Example using the dev JSON file from SQuAD: https://rajpurkar.github.io/SQuAD-explorer/dataset/dev-v2.0.json

enter image description here

Disclaimer: I created Cluria.

user1101329
  • 61
  • 1
  • 2
1

Notepad++ allows you to open one large file at a time.

Try closing all the already opened files in the notepad++ and open this (large) file all alone.

I've faced the same issue, and what I've mentioned above solved my problem.

Edit: If even this doesn't solve your problem, that's a memory issue in your system. Alternatively you can use 010Editor It will allow you to open large files with upto 5GB in space.

Safeer Ansari
  • 772
  • 4
  • 13