1

I am having a problem in accessing the JSONArray ("pages") while reading a json file in Java using the json-simple-1.1.1.jar

My json file is around 32MB of size and the format is as below:

{
    "id": "0912898213",
    "pages": [
        {
            "pageUrl": "http://www.example0.com",
            "results": [
                {
                    "number": "1"
                },
                {
                    "number": "2"
                }
            ]
        },
        {
            "pageUrl": "http://www.example1.com",
            "results": [
                {
                    "number": "3"
                },
                {
                    "number": "4"
                }
            ]
        }
    ]
}

Whereas, the Java code to access this json file is as below:

JSONParser parser=new JSONParser();
JSONObject pagesObject = (JSONObject) parser.parse(new     FileReader(PATH_JSON_DataExtractor));        
JSONArray jsonArray= (JSONArray) pagesObject.get("pages");

for(int i=0; i<jsonArray.size(); i++){}

PROBLEM: The jsonArray is null all the time. Although, the json format is correct and it should work as expected! The above Java code works with the given sample json (also above) but the Java code doesn't work with the 32MB json file. The location of the json file is also correct and the format is also correct, but still I am getting this access issue!

Where I am going wrong to access the json file? I have been looking around on similar questions, and I have followed the exact instructions to access the json file. But I am just lost in getting it right, therefore, looking for suggestions to make this code work. Thank you very much for your time!

Amir
  • 685
  • 3
  • 13
  • 36
  • 1
    Meaby creating a class that represents the JSON object simplifies it to you. parse has a method that get the string and the Class of the object represented. Are you sure its opening the file? Meaby the problem is parsing the content and not opening the file. – aSoler Aug 24 '15 at 11:24
  • No, it's not even reading the "id" element, it returns me null for that as well! – Amir Aug 24 '15 at 11:29
  • Ok. Why not try to read the file and store it in a String, normal way, and then parse? – aSoler Aug 24 '15 at 11:30
  • I shall check with that! – Amir Aug 24 '15 at 11:31
  • But the file size is 32MB, do you think its the good idea? – Amir Aug 24 '15 at 11:37
  • That's true... you can read by lines and when you detect a `}` that closes one object of the array parse that individual object... See http://stackoverflow.com/questions/5868369/how-to-read-a-large-text-file-line-by-line-using-java If your program doesn't need to executes in light systems then meaby is not so creazy to load all of it. – aSoler Aug 24 '15 at 11:41
  • With the json sample you have posted, I don't reproduce the problem: I get `jsonArray.size()`==2. Are you sure the file is read OK? – Little Santi Aug 24 '15 at 11:42
  • @LittleSanti ofcourse this is not the actual json file :D How could I have shown the 32MB file here, whereas, it is only the format of the json important to be known! – Amir Aug 24 '15 at 11:46
  • 1
    Yes, I understand, but I've even multiplied the number of items in your JSON up to 35Mb, and it still works fine. I still suspect the file is not correctly located or updated. Have you tried to modify it (i.e. change id='000') and check if the `pagesObject` is aware of that modification? – Little Santi Aug 24 '15 at 12:08
  • @LittleSanti I really don't get this. The location of the file is correct, format is correct but still it doesn't work, should I share the json file with you? – Amir Aug 24 '15 at 12:27
  • 1
    First try the modification test. – Little Santi Aug 24 '15 at 12:48
  • I partitioned the file into 2 files and executed the same code, it worked! Really thankful to you for your time :) – Amir Aug 24 '15 at 12:56

3 Answers3

3

With the below code it is working perfect for me. Can you check whether the specified file location is correct? Also try reading id like pagesObject.get("id")

package json.simple;

import java.io.FileReader;

import org.json.simple.JSONArray;
import org.json.simple.JSONObject;
import org.json.simple.parser.JSONParser;

public class ReadJSON {
    public static void main(String[] args) throws Exception {
        JSONParser parser = new JSONParser();
        JSONObject pagesObject = (JSONObject) parser.parse(new FileReader("/home/user/tmp/test.json"));
        System.out.println(pagesObject.get("id"));
        System.out.println(pagesObject.get("pages").getClass().getName());
        JSONArray jsonArray= (JSONArray) pagesObject.get("pages");

        for(int i=0; i<jsonArray.size(); i++){
            System.out.println(jsonArray.get(i));
        }
    }
}

And here is the content of my test.json. Exactly same as yours

{
    "id": "0912898213",
    "pages": [
        {
            "pageUrl": "http://www.example0.com",
            "results": [
                {
                    "number": "1"
                },
                {
                    "number": "2"
                }
            ]
        },
        {
            "pageUrl": "http://www.example1.com",
            "results": [
                {
                    "number": "3"
                },
                {
                    "number": "4"
                }
            ]
        }
    ]
}

And here is my dependency in maven

<dependency>
    <groupId>com.googlecode.json-simple</groupId>
    <artifactId>json-simple</artifactId>
    <version>1.1.1</version>
</dependency>
sag
  • 5,333
  • 8
  • 54
  • 91
  • Thanks for the answer. Yea I already checked this as well and it works with this test file, but with the 32MB file it is not working. I believe it is because of the size of the file! How do I deal with it? – Amir Aug 24 '15 at 11:54
  • And my project is a simple Java project, its not a Maven project! – Amir Aug 24 '15 at 12:03
  • 1
    I just tried with 36 MB file and it worked with the same above sample code. Is there any java heap size issue? – sag Aug 24 '15 at 12:27
  • I am not sure about that, it just gives me a NullPointerException at the for loop! – Amir Aug 24 '15 at 12:30
  • 1
    Can you just double check your json. json-simple does not throw error for few json issues. Like, missing comma (,) between array values – sag Aug 24 '15 at 12:52
1

Finally its working now, but I still do not know the actual cause of the problem. I partitioned the file into two separate json files, and executed the code on both of them and it worked. Now I just have to merge them and its done! Not a good solution but couldn't find another way!

Amir
  • 685
  • 3
  • 13
  • 36
0

Use this:-

JSONArray jsonArray= (JSONArray) pagesObject.getJSONArray("pages");

NaveenG
  • 292
  • 3
  • 11
  • Yea I tried this already, I am getting this error "The method getJSONArray(String) is undefined for the type JSONObject" – Amir Aug 24 '15 at 11:49
  • @NaveenG just giving a line of code in this nature does not help with understanding of the problem. It is better to explain what is wrong in the question. – thecoshman Aug 24 '15 at 12:14