Android gson streaming from assets

Question

I have 9 MB txt file with json objects inside it.I need to parse the using gson streaming, i have done


     AssetManager assetManager = activity.getAssets();
            InputStream inputStream;
            inputStream = assetManager.open(fileName);
            JsonReader reader = new JsonReader(new InputStreamReader(inputStream,
                    "UTF-8"));
    JsonElement json = new JsonParser().parse(reader);
                JsonArray array = json.getAsJsonArray();
                Gson gson = new Gson();
                String interationString;
                for (JsonElement jsonElement : array) {
                    interationString = jsonElement.getAsJsonObject().get("drug")
                            .toString();
                    Log.d("drug", interationString);
                    Drug drug = gson.fromJson(interationString, Drug.class);
                    Log.d("drug", "" + drug);
                    DatabaseManager.getInstance().saveDrug(drug);
                }

I have put the json file in assest. But work on device with 4.0 but in emulator 2.3 it was not working and show the logs

05-29 11:19:41.290: D/dalvikvm(164): GC_EXPLICIT freed 74K, 46% free 3344K/6151K, external 1323K/1554K, paused 84ms
05-29 11:19:46.330: D/dalvikvm(373): GC_EXPLICIT freed 2K, 51% free 2718K/5511K, external 716K/1038K, paused 87ms
05-29 11:19:51.370: D/dalvikvm(203): GC_EXPLICIT freed 15K, 51% free 3083K/6215K, external 716K/1038K, paused 102ms
05-29 11:19:56.391: D/dalvikvm(573): GC_EXPLICIT freed 53K, 51% free 2771K/5639K, external 964K/1467K, paused 79ms

Then stop,i have try parse the small file and it was working fine. Can anyone help to parse long file in Android so that it can work in all android device.

Thank you.

possible duplicate of [android: gson performance](http://stackoverflow.com/questions/3261012/android-gson-performance) — Wladimir Palant, May 29 '12 at 09:36

score 6 · Accepted Answer · answered May 29 '12 at 13:19

You'll get the best performance by mixing stream parsing with Gson's data binding. Here's an example that might work for your data set:

public List<Drug> readJsonStream(InputStream in) throws IOException {
    JsonReader reader = new JsonReader(new InputStreamReader(in, "UTF-8"));
    List<Drug> drugs = new ArrayList<Drug>();
    reader.beginArray();
    while (reader.hasNext()) {
        reader.beginObject();
        while (reader.hasNext()) {
            String name = reader.nextName();
            if (name.equals("drug")) {
                Drug drug = gson.fromJson(reader, Drug.class);
                drugs.add(drug);
            } else {
                reader.skipValue();
            }
        }
        reader.endObject();
    }
    reader.endArray();
    reader.close();
    return drugs;
}

Note that calls to beginArray/endArray and beginObject/endObject must be balanced. More details are available on Gson's streaming guide.

+1 - the DOM approach on GSON is quick to implement, but really slow to process large responses. I use it for prototypes, and then optimise down to the Streaming method for production. I think more people need to realise GSON is not so slow, its how you use it (not saying it is the fastest, just that many don't use streaming) — Richard Le Mesurier, Jun 20 '12 at 09:16

score 1 · Answer 2 · answered May 29 '12 at 06:09

1

It look that you are using vanilla XML parse shipped with android - it itilizes kind of DOM model and will slurp complete JSON into object model in memory ( and that's what you see - a lot of memory allocation ).

You should ensure that your parser uses pull model. My choice is GSON, as it has lower memory footpring as jackson. Here is small databinding library I'm using on top of GSON to parse my big JSON data:

https://github.com/ko5tik/jsonserializer

answered May 29 '12 at 06:09

Konstantin Pribluda

12,329
1
30
35

1

No, you are using vanilla JSON parser to slurp full model into memory, and then iterate over it, convert JSON object back to string and parse it with GSON again to get databinding - that' s where your performance is gone – Konstantin Pribluda May 29 '12 at 06:28

Android gson streaming from assets

2 Answers2