Reading large response body as HttpClient

Question

I'm building an android application where I have to read a big chunk of information from a server.

The server sends the information as a JSONArray and everything is fine on the server side.

However the problem comes when I'm trying to receive this JSONArray as it seems to be too large for me to receive?

I connect to the server using HttpClient like this:

HttpParams httpParams = new BasicHttpParams();
int timeoutConnection = 20000;
HttpConnectionParams.setConnectionTimeout(httpParams, timeoutConnection);
int timeoutSocket = 20000;
HttpConnectionParams.setSoTimeout(httpParams, timeoutSocket);

HttpClient client = new DefaultHttpClient(httpParams);
HttpGet request = new HttpGet();
request.setURI(url);

HttpResponse response = null;
try {
    response = client.execute(request);
} catch (ClientProtocolException e) {
    // TODO Auto-generated catch block
    e.printStackTrace();
} catch (IOException e) {
    // TODO Auto-generated catch block
    e.printStackTrace();
}

BufferedReader in = null;
if (response != null) {
    in = new BufferedReader(new InputStreamReader(response.getEntity().getContent()), 100000);
}

return in;

And I simply try to read what's in the BufferedReader with the readLine() method, however what I get back is not the full content of the response. I know this because I tried accessing the same content with a web browser and it returns the full content.

The response from the server is on a single line (might affect it, I don't know)

I'm able to read 4kb and I need to read a whopping 141kb (so I'm far off)..

My personal guess is that somewhere in my code there is a size limit.

Extra I read from the BufferedReader like this:

BufferedReader in = getResponse(new URI(params[0]));
// Read from the received buffer
String s;
StringBuilder sb = new StringBuilder(10000);

if(in != null){
    while ((s = in.readLine()) != null){
        sb.append(s);
    }
}

Log.d("Problem: ", sb.toString());

EDIT: The response does not differ, it's always the same length

EDIT2: It seems that it was Log.d() that had a limit in print length! There was no problem to begin with... sorry!

Because I don't know how to, would it enable me to read a larger response from the server though? Sounds like it would just compress the data. — David Ekermann, Jun 12 '14 at 21:35
I suspect it is your use of BufferedReader. Try [here](http://stackoverflow.com/q/309424/823393) for suggestions about how to convert an InputStream to a String. Better still I suspect your JSON parser will take an InputStream. — OldCurmudgeon, Jun 12 '14 at 22:04
I just noticed that my Log.d might have a size limit... I will look in to this and try to patch my hair back on my head! — David Ekermann, Jun 12 '14 at 22:11

score 1 · Answer 1 · answered Jun 12 '14 at 21:34

1

Use Gzipped JSON in HTTP Request

Checkout this article: http://arnab.ch/blog/2012/09/android-how-to-send-gzipped-json-in-http-request/

answered Jun 12 '14 at 21:34

AAV

3,785
8
32
59

It may reduce the size and improve performance as stated in the article, but I can't see how it would resolve my problem. – David Ekermann Jun 12 '14 at 21:54
Sorry for taking your time it seems that it was Log.d() that had a limit in print length. Thanks for the information though I will look in to Gzip later on to improve the performance. – David Ekermann Jun 13 '14 at 20:14

score 1 · Answer 2 · answered Jun 12 '14 at 21:34

1

Instead of reading the InputStream just directly convert it to String using EntityUtils.toString

example:

 try {
    response = client.execute(request);
    HttpEntity resEntityGet = responseGet.getEntity(); 

    final String result = EntityUtils.toString(resEntityGet); //result is your json response

} catch (ClientProtocolException e) {
    // TODO Auto-generated catch block
    e.printStackTrace();
} catch (IOException e) {
    // TODO Auto-generated catch block
    e.printStackTrace();
}

answered Jun 12 '14 at 21:34

Rod_Algonquin

26,074
6
52
63

Sadly that did not help. – David Ekermann Jun 12 '14 at 21:44
@DavidEkermann then is your response from the server is the problem – Rod_Algonquin Jun 12 '14 at 21:45
I checked the server response with my web browser and it returns the full content as plain text Here take a look: http://213.103.209.154:1234/api/socialCard/fetch/my?token=t79pShThPlgc6P2qJ-3yT5lHpNbB31CUY9_pRel-d9Q – David Ekermann Jun 12 '14 at 21:50
1

@DavidEkermann increase your timeouts and update me. because it took me more than 20seconds just to download the textfile – Rod_Algonquin Jun 12 '14 at 22:14
Sorry for taking your time it seems that it was Log.d() that had a limit in print length. Thanks for the information though it made my code look prettier :) – David Ekermann Jun 13 '14 at 20:13
@DavidEkermann you are welcome.. always pick an answer that helped you and approve it. – Rod_Algonquin Jun 13 '14 at 20:25

Reading large response body as HttpClient

2 Answers2