DefaultHttpClient change response size?

Question

What I try to do

Hello Guys, I'm trying to create an App in which I can view the Orders the Customers gave to me. For this I created a interface on my server, on which I can send post/get/set request's. The response of the Server is in JSON-Format. (For your Information atm only dummydata is filled in)

Now when I do a get request from my app to the server, I get a response from it but it isn't complete about the half of the response I should get isn't there! :( But when I open the URL with the Get-Request in my browser, I get the full response.

Question

Like you see it can't be a server-based problem, because I also tryed via 'curl' to do this get requst, and allways got the full response.

In my App i work with the DefaultHttpClient, so I tought the Problem simply could be that there's a limit for the response but I didn't found it.

So where can I change this "response-size" and what else could be the problem why I don't get the full response! Some good code-snippets or whatever you can imagine would help!

Down here you'll find the code of the Methode which does the Get-Request.

Code

If you need more Code, just write it in the comments!

getOrders()

public void getOrders() {
        Log.d("DataHandlerService", "Aufträge werden geladen");
        Thread t = new Thread() {
            public void run() {

                SharedPreferences settings = getSharedPreferences(PREFS_NAME, 0);
                String userid = settings.getString("userid", "uid");


                Log.d("DataHandlerService", userid);

                // Download-URL
                String URL = "http://api.i-v-o.ch/users/" + userid
                        + "/assignments.json";
                Log.d("Request-URL", URL);

                DefaultHttpClient client = new DefaultHttpClient();
                HttpResponse response;

                try {
                    HttpGet request = new HttpGet();
                    request.setURI(new URI(URL));
                    request.addHeader("Content-Type",
                            "application/x-www-form-urlencoded");
                    response = client.execute(request);


                    int statuscode = response.getStatusLine().getStatusCode();
                    switch (statuscode) {

                    case 200:

                        if (response != null) {

                            StringBuilder sb = new StringBuilder();
                            BufferedReader rd = new BufferedReader(
                                    new InputStreamReader(response.getEntity()
                                            .getContent()));

                            String line;
                            while ((line = rd.readLine()) != null) {
                                sb.append(line + "\n");
                            }
                            String result;
                            result = sb.toString();
                            Log.d("Response", result);
                            JSONReader(result); //here the json will be generated
                        }

                        break;

                    case 500:
                        // Error-Handling
                        break;
                    }
                } catch (Exception e) {
                    e.printStackTrace();
                    Log.e("DataHandler", "URLConnection-Error" + e);
                }

            }
        };
        t.start();
    }

Here's the Response you asked for, like you see a part of it isn't there!:

[{"created_at":"2012-01-06T17:10:00Z","end_datetime":"2008-03-25T13:00:00Z","id":2127,"start_datetime":"2008-03-25T13:00:00Z","updated_at":"2012-01-06T17:10:00Z","title":"2127 Foobar","referee_forename":"Peter","referee_surname":"Gertsch","referee_full_name":"Peter Gertsch","category_title":"Installation - SAT","status_title":"Closed - Erfolgreich"},{"created_at":"2012-01-06T17:10:03Z","end_datetime":"2008-04-04T12:00:00Z","id":2134,"start_datetime":"2008-04-04T12:00:00Z","updated_at":"2012-01-06T17:10:03Z","title":"2134 Foobar","referee_forename":"Daniel","referee_surname":"Brunner","referee_full_name":"Daniel Brunner","category_title":"Installation - SAT","status_title":"Closed - Erfolgreich"},{"created_at":"2012-01-06T17:10:03Z","end_datetime":"2008-04-07T12:00:00Z","id":2136,"start_datetime":"2008-04-07T12:00:00Z","updated_at":"2012-01-06T17:10:03Z","title":"2136 Foobar","referee_forename":"Andreas","referee_surname":"Lutz","referee_full_name":"Andreas Lutz","category_title":"Installation - SAT","status_title":"Closed - technisches problem"},{"created_at":"2012-01-06T17:10:08Z","end_datetime":"2008-05-22T07:00:00Z","id":2144,"start_datetime":"2008-05-22T07:00:00Z","updated_at":"2012-01-06T17:10:08Z","title":"2144 Foobar","referee_forename":"Pascal","referee_surname":"Pichand","referee_full_name":"Pascal Pichand","category_title":"Installation - SAT","status_title":"Closed - Erfolgreich"},{"created_at":"2012-01-06T17:10:08Z","end_datetime":"2008-05-15T07:00:00Z","id":2145,"start_datetime":"2008-05-15T07:00:00Z","updated_at":"2012-01-06T17:10:08Z","title":"2145 Foobar","referee_forename":"Hansruedi","referee_surname":"W\u00fcrgler","referee_full_name":"Hansruedi W\u00fcrgler","category_title":"Installation - SAT","status_title":"Closed - Erfolgreich"},{"created_at":"2012-01-06T17:10:08Z","end_datetime":"2008-05-26T08:00:00Z","id":2146,"start_datetime":"2008-05-26T08:00:00Z","updated_at":"2012-01-06T17:10:08Z","title":"2146 Foobar","referee_forename":"Martina","referee_surname":"Issler","referee_full_name":"Martina Issler","category_title":"Installation - SAT","status_title":"Closed - Erfolgreich"},{"created_at":"2012-01-06T17:10:08Z","end_datetime":"2008-06-03T14:00:00Z","id":2147,"start_datetime":"2008-06-03T14:00:00Z","updated_at":"2012-01-06T17:10:08Z","title":"2147 Foobar","referee_forename":"Matthias ","referee_surname":"Kuhn","referee_full_name":"Matthias  Kuhn","category_title":"Installation - SAT","status_title":"Closed - Erfolgreich"},{"created_at":"2012-01-06T17:10:12Z","end_datetime":"2008-07-07T07:00:00Z","id":2157,"start_datetime":"2008-07-07T07:00:00Z","updated_at":"2012-01-06T17:10:12Z","title":"2157 Foobar","referee_forename":"Eberhard","referee_surname":"Polatzek","referee_full_name":"Eberhard Polatzek","category_title":"Installation - SAT","status_title":"Closed - Erfolgreich"},{"created_at":"2012-01-06T17:10:13Z","end_datetime":"2008-07-11T08:00:00Z","id":2161,"start_datetime":"2008-07-11T08:00:00Z","updated_at":"2012-01-06T17:10:13Z","title":"2161 Foobar","referee_forename":"Magali","referee_surname":"Bohin","referee_full_name":"Magali Bohin","category_title":"Installation - SAT","status_title":"Closed - Erfolgreich"},{"created_at":"2012-01-06T17:10:14Z","end_datetime":"2008-07-25T08:30:00Z","id":2163,"start_datetime":"2008-07-25T08:30:00Z","updated_at":"2012-01-06T17:10:14Z","title":"2163 Foobar","referee_forename":"(Hotel Centrum Griesalp)","referee_surname":"Haltenegg Betriebs AG","referee_full_name":"(Hotel Centrum Griesalp) Haltenegg Betriebs AG","category_title":"Installation - SAT","status_title":"Closed - Erfolgreich"},{"created_at":"2012-01-06T17:10:16Z","end_datetime":"2008-08-07T09:00:00Z","id":2170,"start_datetime":"2008-08-07T09:00:00Z","updated_at":"2012-01-06T17:10:16Z","title":"2170 Foobar","referee_forename":".","referee_surname":"SAC Hollandiah\u00fctte","referee_full_name":". SAC Hollandiah\u00fctte","category_title":"Installation - SAT","status_title":"Closed - Erfolgreich"},{"created_at":"2012-01-06T17:10:16Z","end_datetime":"2009-05-07T06:30:00Z","i

**FYI**: The way you are reading your `HttpResponse` is pretty much bonkers. Use `EntityUtils.toString(HttpEntity)` instead. — Jens, Feb 08 '12 at 08:00
hey Jens, i tried this like you told me i found some stuff out first when I get the length of the response over 'response.getEntity().ContentLength();' I get 7046, but when I check how long the response is after I made it to a String its only about 4060, how can I solve this?? — safari, Feb 08 '12 at 08:42
byte size does not always correlate to string size unless you only have US-ASCII/Latin-1 or other 1-byte character sets in your JSON. what type of string content are you receiving (i.e. can you post a JSON sample output). — Jens, Feb 08 '12 at 09:43
its encoded in UTF-8 i'll add the json i'll get over the response, if you check the link http://www.i-v-o.ch/users/116/assignments.json you get the whole json. I'll post you the response I get! — safari, Feb 08 '12 at 10:41

score 3 · Accepted Answer · answered Feb 08 '12 at 11:43

Ah. Right, the problem isn't your connection or anything like that. Your service is returning an array - not an object - thus you should parse it like this:

HttpResponse response = ...
if (.. validate status ..) {
    JSONArray array = new JSONArray(HttpEntityUtils.toString(response.getEntity()));
    // Your JSONArray is now ready to play with.
}

And consider using an AsyncTask instead of a Thread, like this:

class AssignmentsTask extends AsyncTask<String, Void, JSONArray> {
    @Override
    protected JSONArray doInBackground(String... params) {
        final String url = "http://api.i-v-o.ch/users/" + params[0]
                + "/assignments.json";
        try {
            HttpResponse response = mClient.execute(new HttpGet(url));
            if (response.getStatusLine().getStatusCode() == 200) {
                return new JSONArray(EntityUtils.toString(response.getEntity()));
            } else {
                Log.w(TAG, "Error receiving assignments for " + params[0] + ", " + response.getStatusLine());
            }
        } catch (ClientProtocolException e) {
            Log.w(TAG, "Proto: Error fetching assignments for " + params[0], e);
        } catch (IOException e) {
            e.printStackTrace();
            Log.w(TAG, "IO: Error reading assignments for " + params[0], e);
        } catch (ParseException e) {
            Log.w(TAG, "Parse: Error parsing assignments for " + params[0], e);
        } catch (JSONException e) {
            Log.w(TAG, "JSON: Error parsing JSON for " + params[0], e);
        }
        return null;
    }

    @Override
    protected void onPostExecute(JSONArray result) {
        // Stuff that handles the resulting JSONObject on
        // the UI-thread goes here (i.e. update View:s)

        // result is null if the operation failed
    }
}

And to retrieve an order for the user "116":

new AssignmentsTask().execute("116");

i get this error :( '02-08 15:41:27.960: W/System.err(31510): java.lang.IllegalStateException: Content has been consumed' because of this line 'JSONArray array = new JSONArray(HttpEntityUtils.toString(response.getEntity()));' :( — safari, Feb 08 '12 at 14:46
well, you only get to consume (i.e. read the contents of the stream) the HttpEntity once, just like in the example above. Calling EntityUtils.toString(HttpEntity) will consume the content and return the response as a String to the JSONArray:s constructor. — Jens, Feb 08 '12 at 19:34

score 0 · Answer 2 · edited May 23 '17 at 12:16

The response size should be given by the web server you are contacting. You could read the response size using :

httpResponse.getEntity().getContentLength()

Also, what can happen is a connection timeout, making it impossible for the client to receive all data of the response. In that case, try using a timeout that is long enough to be sure you get all the data.

If your json is too large, then it's not a good idea in a mobile context to expect all the data coming in a single request, you could then have to design a web server that could give you chunks of a response, you would then require the first chunk, then the a different one, etc..

Usually, the http protocole's partial content is the answer for that problem.

DefaultHttpClient change response size?

2 Answers2

Linked