2

My Android app requests information from our server using an HttpURLConnection object. The server returns some text -- could be a little (~50 characters) or a lot (12 MB) depending on the request. The mime type is text/plain.

Upon my initial implementation, I was disappointed to discover that getInputStream() was returning the entire HTTP response from the server -- including the headers -- even though the documentation said it contains only the response body. Oh well... I implemented a little state machine to read and store the headers and provided interfaces in my code to allow callers to access header fields. Called it from an AsyncTask to get it off the UI thread and to get an easy progress tracking method. All was well.

I've recently added another feature that requests information from the server, so I used my same "http connection wrapper" object, which expects to see headers in the body stream. This time, it was convenient to use a Thread object to push the download off the UI thread. (I didn't need the progress tracking and ironically, Thread is easier to implement than the "convenience object" AsyncTask.) Interestingly, I was not seeing the headers in the getInputStream() stream when called from a Thread object.

The only difference is that in one case I'm making the request from inside an object derived from AsyncTask and in the other case I'm making the request from inside an object derived from Thread.

When I use HttpUrlConnection from an AsyncTask, I see headers in the input stream. When I use HttpUrlConnection from a Thread, I see only the response body. The latter is the "correct" behavior according to the documentation.

I've tried switching methods. That is, I make my first request in a Thread instead of an AsyncTask. If I do that, I do not see the headers. When I make my second request in an AsyncTask instead of a Thread, I see the headers in the input stream where only the body is expected. So the issue seems to be related to whether I'm in an AsyncTask or in a Thread.

Obviously I could choose one or the other methods of getting my HTTP activity off the UI thread, but I'd like to understand what's going on so I can make it behave identically regardless of how it's called. And I like using AsyncTask because it has an easy built-in progress tracking mechanism, but that's the method that isn't working right.

Ideas? Suggestions?

MORE INFORMATION

I've encapsulated my HttpURLConnection access in a little function that looks like this (the actual code has a delegate to which it sends the received string, but I've simplified it for testing):

/**
 * Reads the data at the URL.
 * @param urlString The URL to read
 */
public void Get(
    String urlString)
    {
    URL url;
    BufferedInputStream in;
    try
        {
        url = new URL(urlString);
        conn = (HttpURLConnection) url.openConnection();
        conn.setRequestMethod("GET");
        conn.setRequestProperty("Accept", "text/*");
        in = new BufferedInputStream(conn.getInputStream(), 8192);

        byte [] buffer = new byte[8192];
        StringBuilder stringBuffer = new StringBuilder(16384);

        int read;
        while ((read = in.read(buffer)) != -1)
            {
            String tempstr = new String(buffer, 0, read, "ISO-8859-1");
            System.out.println(tempstr);
            }
        return true;
        }
    catch (IOException e)
        {
        return false;
        }

I call this function from two places. For testing, I have both places requesting the same page from our server. First is in an object that extends AsyncTask. In doInBackground(), I instantiate the object that contains Get(), then call Get(). Second is in an object that extends Thread, where I instantiate the object that contains Get() then call Get() in run(). In both cases I request the same page.

The result is when I call from my AsyncTask, I see ALL the response headers. What I actually see is the raw HTTP -- including chunk lengths when Transfer-Encoding is "chunked". When I call from my Thread, I get just the body of the HTML, which is what I expect.

For testing, I wrote a script on the server that returns the raw request. Here's what I get (note "awebsite" is not the name of my server):

When running in a Thread:

Connection: Keep-Alive
Accept: text/*
Host: www.awebsite.com
User-Agent: Java0

When running in an AsyncTask:

[CRLF]
HTTP/1.1 200 OK
Date: Tue, 31 May 2011 23:29:20 GMT
Server: Microsoft-IIS/6.0
X-Powered-By: ASP.NET
Content-Type: text/html
Set-Cookie: ASPSESSIONIDCQSTTQAQ=OHBIFGJCPEAAHGNHICPKOKFO; path=/
Cache-control: private
Transfer-Encoding: chunked
[CRLF]
53
Connection: Keep-Alive
Accept: text/*
Host: www.awebsite.com
User-Agent: Java0
[CRLF]
0
[CRLF]

Note that both situations seem to send the same request. However, something is tricking HttpURLConnection into giving me the response headers right along with the body in the case where I call Get() from an AsyncTask.

To further confuse things, accessing www.google.com works as expected -- I never see headers mixed with the body. So that would seem to indicate there's either something wrong on the server side (which makes me wonder how the server knows to fail) or there's something about how the server responds that confuses HttpURLConnection when it's running in an AsyncTask.

Craig
  • 3,253
  • 5
  • 29
  • 43
  • I face the same. I send requests from Thread and SOMETIMES all headers are there in getInputStream(). Also looking for a way to fix it. – Fedor Jun 29 '11 at 01:41
  • @Chirag, would you please tell me why should one use headers. I have to validate some credentials from android I am using php on xammp. how should i go for it. as i don't know how to write php code with headers – Pankaj Nimgade Feb 17 '15 at 08:04

2 Answers2

3

I expect there is an error in your server side generation: perhaps there is a response header that has a couple of embedded newlines and the http parser is taking that to be the end of the headers.

Femi
  • 64,273
  • 8
  • 118
  • 148
  • How would the server know whether I'm requesting the page from a Thread or an AsyncTask? The data is the same (good, bad, malformed, whatever) in either case. – Craig May 31 '11 at 15:02
  • I think Femi is right. Are you sure that you request the same thing from the server in both cases? I never had the issue that the response contained the headers as you described. Try to request something from a public domain and see if you get the header from them, too (use microsoft, google, yahoo, apple... try them all :) ) – WarrenFaith May 31 '11 at 15:33
  • Well, the server CAN'T know if you're in a Thread or an AsyncTask, so you must be either requesting something different or using something like pipelining or keepalives incorrectly. Paste the 2 chunks of code that you are using. – Femi May 31 '11 at 15:58
  • Further information: The problem does appear to be isolated to our server, but that doesn't explain anything. If I request www.google.com I never see the headers. But if I request a page on our server, I get the headers in the body stream if I request it from AsyncTask, but only the body content if I request it from a Thread. The code that actually does the request and reads the response is the same regardless of the thread it's running in. – Craig May 31 '11 at 19:52
  • So your server side code is writing an incorrect header: if you have access to **wget** and you can connect to the URL without authentication you can do `wget -S -O - "{url}"` and see what the headers are on a request that doesn't work correctly. – Femi May 31 '11 at 19:56
  • Stupid comment editor. Well, this is what you get: `URL url = new URL(urlString);HttpURLConnection conn = (HttpURLConnection) url.openConnection();BufferedInputStream in = new BufferedInputStream(conn.getInputStream(), 8192); byte [] buffer = new byte[8192]; int read; while ((read = in.read(buffer)) != -1) { System.out.println(new String(buffer,0,read,"ISO-8859-1"); }` – Craig May 31 '11 at 19:57
  • Again, how does the server know that I'm accessing through AsyncTask and therefore it should write a wrong header? And if it can't know that (it can't) what is HttpURLConnection doing when running in an AsyncTask that causes it to tell the server to send it a bad header, or that tells it to misinterpret the headers it sees? – Craig May 31 '11 at 20:01
  • Does the `wget` call produce the same results? – Femi May 31 '11 at 20:19
  • I'm using `curl -S -0 -I "url"` on my Mac and I see reasonable headers -- imilar to any other site. Without the -I I get just the body content -- no headers. With -i I get the headers followed by the body content. All looks fine to me. Again, since the server doesn't know whether I'm using an AsyncTask or a Thread in my Android code, this is what I would expect to see -- that is, the server is producing the same, reasonable output and it's being misinterpreted by HttpURLConnection -- hence my confusion. – Craig May 31 '11 at 20:37
  • Interesting: post the header output? HttpURLConnection has historically worked fine, and there is no way for the server to tell how it is being requested, so that really can't be it. Unless Java is sending some request header that is misleading the server into producing garbage output. – Femi May 31 '11 at 20:46
  • @Femi and @WarrenFaith: I've updated the question with code and test results. Thanks for any help you can offer. – Craig Jun 01 '11 at 11:20
  • What happens if you turn off Keep-Alive on your server? Close the connection on each request? – Femi Jun 01 '11 at 13:14
  • If I send Connection:close with the request, I get back Connection:close from the server instead of Connection:Keep-Alive. Otherwise, the behavior is the same. I really think we're looking in the wrong place. It's obvious that the server is behaving the same way every time and that it's HttpURLConnection that is misbehaving. I think it's probably more related to how Thread.run() and AsyncTask.doInBackground() work. – Craig Jun 01 '11 at 18:20
  • Alright. It IS possible you've found a thread specific bug buried deep in the HttpURLConnection code, but given that on Android that uses Apache HTTP client under the hood I'd be VERY surprised if that's the case. HttpURLConnection is pretty battle-hardened. – Femi Jun 01 '11 at 18:32
  • I agree my server is playing a role, but I can't find any indication that the server is sending something different under AsyncTask than under Thread. It's possible, though, that something it sends **exposes** the problem in AsyncTask and/or HttpURLConnection. I think the solution may be to re-write my AsyncTask to use Thread or re-write my HttpURLConnection wrapper object to use HttpClient. – Craig Jun 01 '11 at 19:50
  • @Femi, would you please tell me why should one use headers. I have to validate some credentials from android I am using php on xammp. how should i go for it. as i don't know how to write php code with headers – Pankaj Nimgade Feb 17 '15 at 08:06
0
  1. Check conn.getResponseCode(). If it's -1 you should not try to read the stream.
  2. If you're on 2.1 or lower do the following: System.setProperty("http.keepAlive", "false");
    More details here HttpURLConnection.getResponseCode() returns -1 on second invocation.
Community
  • 1
  • 1
Fedor
  • 43,261
  • 10
  • 79
  • 89
  • would you please tell me why should one use headers. I have to validate some credentials from android I am using php on xammp. how should i go for it. as i don't know how to write php code with headers – Pankaj Nimgade Feb 17 '15 at 08:05