5

I am receiving huge JSON and and while I am reading the lines OutOfMemoryError appears.

Here is my first method that I am trying to parse the JSON.

    InputStream in = response.getEntity().getContent();
    BufferedReader reader = new BufferedReader(new InputStreamReader(in, "UTF-8"), 8);
    String result = "";

    while (true) {

                String ss = reader.readLine();
                if (ss == null) {
                        break;
                }
                result += ss;
   }

And I've also tried this method.

    InputStream in = response.getEntity().getContent();
    BufferedReader reader = new BufferedReader(new InputStreamReader(in, "UTF-8"), 8);
    StringBuilder sb = new StringBuilder();
    String line = null;

    while ( (line = reader.readLine()) != null)
    {
    sb.append(line);
    }

In the both of the cases the OutOfMemoryErorr is appearing.

Naskov
  • 4,121
  • 5
  • 38
  • 62
  • Did you try using the android standard solution for parsing JSON files? http://developer.android.com/reference/org/json/package-summary.html, also have a look at JsonReader: http://developer.android.com/reference/android/util/JsonReader.html – sulai Jan 28 '13 at 14:34
  • What is the size of incoming data? – Leonidos Jan 28 '13 at 14:44
  • It's around 10mb per Object and in the JSON are more than 20 Objects, it's 10mb cause I am getting base64.. – Naskov Jan 28 '13 at 14:47
  • And I've tried the Standard JSON parsers but they are not working. – Naskov Jan 28 '13 at 14:47

4 Answers4

3

Mostly the error will occur due to heap size. You need to increase the size of the heap

To increase the heap size

For additional info on heap size in java visit here

Community
  • 1
  • 1
Raja Asthana
  • 2,080
  • 2
  • 19
  • 35
  • this is wrong. he wont be able to change heap size on a real device. And I'm not sure that -Xmx2048m option will change anything in android. – Leonidos Jan 28 '13 at 14:43
  • I can't find the "Arguments" option in Run Configurations. I am using Android Development Tools (Eclipse). – Naskov Jan 28 '13 at 14:50
3

The best solution I found was to raise the Heap of the Application.

I placed android:largeHeap="true" under the <application/> in the AndroidManifest.xml.

Naskov
  • 4,121
  • 5
  • 38
  • 62
  • `android:largeHeap=true` is not the recommended solution: [https://developer.android.com/training/articles/memory.html#YourApp](https://developer.android.com/training/articles/memory.html#YourApp) – Ken Tan Feb 01 '16 at 16:51
  • @KenTan the answer is from 2013, now it's 2016 :) – Naskov Feb 02 '16 at 05:05
2

When you instantiate your BufferedReader, you pass in an int size of 8, meaning your buffer is 8 characters (I assume that's not what you want). If you pass no size, the default is 8192 characters.

Try this:

BufferedReader reader = new BufferedReader(new InputStreamReader(in, "UTF-8"));

Or this (same effect):

BufferedReader reader = new BufferedReader(new InputStreamReader(in, "UTF-8"), 8192);

http://developer.android.com/reference/java/io/BufferedReader.html

I'm not completely sure that will fix the error you are seeing, but worth trying if you can't get it working.

Steven Byle
  • 13,149
  • 4
  • 45
  • 57
  • I am using BufferedReader reader = new BufferedReader(new InputStreamReader(in, "UTF-8"), 8192); and there is still Out Of Memory. – Naskov Jan 28 '13 at 14:48
  • 1
    Hmm... I'm reading above you saying that you have 20 objects at 10MB a piece? If that's the case, that's a payload of 200MB that you are trying to hold in memory (hence OOM errors). Even 10MB is a huge payload for JSON, as many devices have an app heap size around 20-30MB. You may want to look into chunking your download or breaking up the JSON that the service is returning. – Steven Byle Jan 28 '13 at 15:00
  • I was hoping that would be the worst case scenario, but it seems that is the only way. Thanks Sir. – Naskov Jan 28 '13 at 15:07
1

My JSON code was waiting for status, which comes towards the end. So I modified the code to return earlier.

// try to get formattedAddress without reading the entire JSON
        String formattedAddress;
        while ((read = in.read(buff)) != -1) {
            jsonResults.append(buff, 0, read);
            formattedAddress = ((String) ((JSONObject) new JSONObject(
                    jsonResults.toString()).getJSONArray("results").get(0))
                    .get("formatted_address"));
            if (formattedAddress != null) {
                Log.i("Taxeeta", "Saved memory, returned early from json") ;
                return formattedAddress;
            }               
        }
JSONObject statusObj = new JSONObject(jsonResults.toString());
        String status = (String) (statusObj.optString("status"));
        if (status.toLowerCase().equals("ok")) {
            formattedAddress = ((String) ((JSONObject) new JSONObject(
                    jsonResults.toString()).getJSONArray("results").get(0))
                    .get("formatted_address"));
            if (formattedAddress != null) {
                Log.w("Taxeeta", "Did not saved memory, returned late from json") ;
                return formattedAddress;
            }           
        } 
Siddharth
  • 9,349
  • 16
  • 86
  • 148