Android HttpClient Outof memory Error (xml data)

Question

I am having a problem to read large data from xml web service the xml file is about 5.5Mb and the code crashes and raise out of memory error

Here is my function

 private static HttpResponse executePostHttpRequest(String baseUrl,
  String names[],
  String values[]) throws ClientProtocolException,
  IOException {
final HttpClient client = newHttpClientInstance();
HttpPost request = new HttpPost(baseUrl);

boolean haveData = (names != null) && (values != null);

// if we have data, form it into request
if (haveData) {

  List<NameValuePair> nameValuePairs = new ArrayList<NameValuePair>(values.length);

  for (int i = 0; i < values.length; i++) {
    nameValuePairs.add(new BasicNameValuePair(names[i], values[i]));
  }

  try {
    request.setEntity(new UrlEncodedFormEntity(nameValuePairs, "utf-8"));
  } catch (UnsupportedEncodingException e) {
    request.setEntity(new UrlEncodedFormEntity(nameValuePairs));
  }
}

request.addHeader("Content-Type", "application/x-www-form-urlencoded;charset=utf-8");
// return response created by executing this request
return client.execute(request);

}

and here is my dumb

        12-05 23:30:31.813: I/dalvikvm-heap(443): Forcing collection of SoftReferences for 4842758-byte allocation
    12-05 23:30:31.964: D/dalvikvm(443): GC freed 0 objects / 0 bytes in 143ms
    12-05 23:30:31.964: E/dalvikvm-heap(443): Out of memory on a 4842758-byte allocation.
    12-05 23:30:31.964: I/dalvikvm(443): "pool-1-thread-3" prio=5 tid=35 RUNNABLE
    12-05 23:30:31.977: I/dalvikvm(443):   | group="main" sCount=0 dsCount=0 s=N obj=0x44f92608 self=0x3d92a0
    12-05 23:30:31.977: I/dalvikvm(443):   | sysTid=460 nice=0 sched=0/0 cgrp=default handle=3901200
    12-05 23:30:31.984: I/dalvikvm(443):   at java.lang.AbstractStringBuilder.enlargeBuffer(AbstractStringBuilder.java:~97)
    12-05 23:30:31.984: I/dalvikvm(443):   at java.lang.AbstractStringBuilder.append0(AbstractStringBuilder.java:155)
    12-05 23:30:31.984: I/dalvikvm(443):   at java.lang.StringBuilder.append(StringBuilder.java:216)
    12-05 23:30:31.984: I/dalvikvm(443):   at com.XXXXXXXXX.api.HttpUtils.convertStreamToString(HttpUtils.java:308)
    12-05 23:30:31.993: I/dalvikvm(443):   at com.XXXXXXXXX.api.HttpUtils.responseToString(HttpUtils.java:331)
    12-05 23:30:31.993: I/dalvikvm(443):   at com.XXXXXXXXX.api.HttpUtils.executeRequest(HttpUtils.java:208)
    12-05 23:30:31.993: I/dalvikvm(443):   at com.XXXXXXXXX.api.HttpUtils.access$0(HttpUtils.java:188)
    12-05 23:30:31.993: I/dalvikvm(443):   at com.XXXXXXXXX.api.HttpUtils$3.run(HttpUtils.java:171)
    12-05 23:30:31.993: I/dalvikvm(443):   at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1068)    
12-05 23:30:31.993: I/dalvikvm(443):   at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:561)
12-05 23:30:31.993: I/dalvikvm(443):   at java.lang.Thread.run(Thread.java:1096)
12-05 23:30:33.024: D/HttpUtils(443): Throwable: java.lang.OutOfMemoryError
12-05 23:30:35.383: D/HttpUtils(443): Throwable: java.lang.OutOfMemoryError

Any advice?

For Android 3.0+, you can try [android:largeHeap="true"](http://developer.android.com/reference/android/content/pm/ApplicationInfo.html#FLAG_LARGE_HEAP) — yorkw, Dec 06 '11 at 00:00

score 2 · Accepted Answer · edited May 23 '17 at 12:11

2

Zip the xml, text compresses very well, I had a similar issue (JSON instead of XML), not only did it solve my issues with memory, it also made the app a million times faster.

In my specific case, the JSON string changed from 5 Mb, to 200kb.

Look in this direction: Gzip in Android

edited May 23 '17 at 12:11

Community

1
1

answered Dec 05 '11 at 23:47

blindstuff

18,298
10
47
48

Is zipping really going to help him though? Isn't he going to need to unzip it in the end then it will just balloon up to 5MB in memory and give him another out of memory error again? – Kurtis Nusbaum Dec 06 '11 at 01:01
Thank you for help But your solution requires me to modify alot of code that depends on my httpUtils class , so i decided to use XMLPullParser to read Xml Stream dirictly aand baypassing the httpUtils Class . Thanks again – Ahmed Hussein Dec 06 '11 at 04:28

score 0 · Answer 2 · edited Apr 20 '23 at 05:24

Try reading the actual input stream directly. This will allow you to use an XMLPullParser, meaning you won't have to keep the entire XML file in memory while you're parsing it. Pull parsing works by parsing the element as soon as you get it and relying on state (like a SAX parser) to keep track of where you are. It's a need concept that lets you think of you xml file as a stream of XML tags coming in.

You'd wanna do something like the following:

URL url = new URL("http://www.android.com/");
HttpURLConnection urlConnection = (HttpURLConnection) url.openConnection();
try {
  InputStream in = new BufferedInputStream(urlConnection.getInputStream());
  XmlPullParserFactory factory = XmlPullParserFactory.newInstance();
  factory.setNamespaceAware(true);
  XmlPullParser parser = factory.newPullParser(); 
  parser.setInput(new InputStreamReader(in));
  XmlUtils.beginDocument(parser,"results");
  int eventType = parser.getEventType();
  do{
    XmlUtils.nextElement(parser);
    parser.next();
    eventType = parser.getEventType();
    if(eventType == XmlPullParser.TEXT){
      Log.d("test",parser.getText());
    }
    //...Handle other types of events...
  } while (eventType != XmlPullParser.END_DOCUMENT) ;
}
finally {
  urlConnection.disconnect();
}

I cobled that example together from this tutorial and this documentation. I've haven't actually tried it to see if it works though.

Android HttpClient Outof memory Error (xml data)

2 Answers2