InputStream and OutOfMemory Error

Question

public String loadJSONFromAsset(String path) {
    String json = null;
    try {
        InputStream is = this.getAssets().open(path);
        int size = is.available();
        Log.d("Size: ", "" +size);
        byte[] buffer = new byte[size];
        is.read(buffer);
        is.close();
        json = new String(buffer, "UTF-8");
    } catch (IOException ex) {
        ex.printStackTrace();
    }
    return json;
}

This is code which convert the file to JSON data file. It works literally, it creates JSON file but the size of "is" is appr. 8MB

D/Size:: 7827533

and OutOfMemory Error occurs at most devices such as

java.lang.OutOfMemoryError
at java.lang.String.<init>(String.java:255)
at java.lang.String.<init>(String.java:228)
at com.example.fkn.projecttr.List.loadJSONFromAsset(List.java:255)

How can I handle it? How can it be coded more efficient? It has no problem running time but it consumes too large memory on device. Thus, when the device memory has no more capacity, program crashes out.

Unrelated to your issue, but don't `return` from a `catch` block. — bradimus, Jul 22 '16 at 14:47
This might not be happening as a result of what you're doing here: objects created in other code could be taking up most of your available memory. — Andy Turner, Jul 22 '16 at 14:50
add android:largeHeap="true" in the AndroidManifest application tag — X3Btel, Jul 22 '16 at 14:55
@bradimus is it for styling reasons or there is some risk returning from catch block? — lelloman, Jul 22 '16 at 14:56
@lelloman There is the "one return point" school of thinking (which I am not a subscriber of), that implies that you cannot return from a catch block since you'll also have a "normal" return path. See also http://stackoverflow.com/questions/36707/should-a-function-have-only-one-return-statement — Durandal, Jul 22 '16 at 15:39
@lelloman It is more than styling. A `return` from a `catch` block may not actually return. See here: http://stackoverflow.com/questions/2309964/multiple-returns-which-one-sets-the-final-return-value — bradimus, Jul 22 '16 at 17:04
@Durandal bradimus thanks for the input, now I realize that I've always been a hippie about this >_ — lelloman, Jul 22 '16 at 18:41

score 1 · Answer 1 · edited May 23 '17 at 12:22

I noticed this:

    int size = is.available();

and thought it was a little strange. So I went and looked at the JavaDoc for InputStream.available and this is what it had to say:

Note that while some implementations of InputStream will return the total number of bytes in the stream, many will not. It is never correct to use the return value of this method to allocate a buffer intended to hold all data in this stream.

So you have one of two conditions:

Your file size is actually 8MB.

If you really have this much JSON, you need to rethink what is in there and what you are using it for. One option that I don't see a lot of developers use is JsonReader, which allows you to parse through the JSON without loading the entire stream into memory first.
Your file size is much smaller than 8MB

Just read the file differently, see How do I create a Java string from the contents of a file?

score 0 · Answer 2 · answered Jul 22 '16 at 15:16

Keep the bytes compressed, a GZipOutputStream on a ByteArrayOutputStream (or a gzipped compressed asset).

Then always process that, for output, parsing or whatever, using a GZipInputStream.

That would safe at least a factor 20 (10 compression, ~3 times less overhead).

For a less invasive change (and more memory consumption): now there is an 8 MB byte array and a 16 MB String. The string could be immediately parsed to a DOM, so JSON. Discarding the whitespace, and equal String values mapped to one String instance (for instance by a Map<String, String> idmap). It depends whether there is much repetition in the data.

InputStream and OutOfMemory Error

2 Answers2