Android HttpUrlConnection EOFException

Question

I would like to know if there are known issues on Android with HttpUrlConnection and POST requests. We are experiencing intermittent EOFExceptions when making POST requests from an Android client. Retrying the same request will eventually work. Here is a sample stack trace:

java.io.EOFException
at libcore.io.Streams.readAsciiLine(Streams.java:203)
at libcore.net.http.HttpEngine.readResponseHeaders(HttpEngine.java:579)
at libcore.net.http.HttpEngine.readResponse(HttpEngine.java:827)
at libcore.net.http.HttpURLConnectionImpl.getResponse(HttpURLConnectionImpl.java:283)
at libcore.net.http.HttpURLConnectionImpl.getResponseCode(HttpURLConnectionImpl.java:497)
at libcore.net.http.HttpsURLConnectionImpl.getResponseCode(HttpsURLConnectionImpl.java:134)

There are many similar bug reports and posts to stack overflow but I cannot understand if there really is an issue and if so, what versions of Android are affected and what the proposed fix/work around is.

Here are some of the similar reports I am referring to:

Here is a potential Android framework fix

https://android.googlesource.com/platform/libcore/+/19aa40c81c48ff98ccc7272f2a3c41479b806376

I do know there was an issue with poisoned connections in the connection pool in pre-Froyo but these issues are occurring on new ICS+ devices exclusively. If there were a problem on later devices I would expect some kind of official Android documentation of the issue.

What about this work-around? Is there anything wrong with it? http://stackoverflow.com/a/17638671/609782 — Darpan, Aug 13 '15 at 12:37
@Darpan you could just try it, although it seems unrelated based on the stack trace.. — Kevin, Apr 22 '16 at 07:16

score 12 · Accepted Answer · answered Nov 30 '13 at 16:40

12

Our conclusion is that there is an issue in the Android platform. Our workaround was to catch the EOFException and retry the request N number of times. Below is the pseudo code:

private static final int MAX_RETRIES = 3;

private ResponseType fetchResult(RequestType request) {
    return fetchResult(request, 0);
}

private ResponseType fetchResult(RequestType request, int reentryCount) {
    try {
        // attempt to execute request
    } catch (EOFException e) {
        if (reentryCount < MAX_RETRIES) {
            fetchResult(request, reentryCount + 1);
        }
    }
    // continue processing response
}

answered Nov 30 '13 at 16:40

Matt Accola

4,090
4
28
37

2

Hi Matt. I am trying this but it is not working. I want to verify that your pseudo code is as follows: 1. open connection via url.openConnection(). 2. If I get a EOF when calling getResponseCode(), then disconnect, set HttpURLConnection to null and connect again. 3. Try this until max number of retries. Is this what you did? Thank you. – Juan Acevedo Dec 04 '13 at 03:10
1

Yes, each time you try the request I would advise you to recreate the HttpURLConnection, don't try to reuse the same instance. When you say it is not working what do you mean? – Matt Accola Dec 15 '13 at 15:05
1

Thanks for answering. I meant I tried doing as you said and I am still having problems with EOFException. – Juan Acevedo Dec 16 '13 at 20:05
Can you post more details on what problems you are having. Do you just repeatedly get an EOFException no matter how many times you try it? What device(s)/versions of Android are you testing on? – Matt Accola Dec 18 '13 at 07:21
It is working now without code change, weird... But now my SSL Engine is crapping out. Different issue, but thanks for the help. – Juan Acevedo Dec 18 '13 at 20:34
This approach is a nice start but it doesn't always work. A reliable alternative uses setRequestProperty("Connection", "close") only when needed, combined with a retry count that reflects the maximum possible number of stale connections. See my answer below, here http://stackoverflow.com/a/23795099/204480. – James Wald May 22 '14 at 02:10

score 7 · Answer 2 · answered Apr 03 '14 at 07:04

HttpURLConnection library internally maintains a pool of Connections. So, whenever a request is send, it first checks if there is an existing connection already present in the pool, based on which it decides to create a new one.

These connections are nothing but sockets, and this library by default does not closes these sockets. It may sometimes happen that a connection (socket) which is not currently being used and is present in the pool is no longer usable as the Server may choose to terminate the connection after some time. Now, since the connection even though is closed by the server, the library does not knows about it and assumes the connection/socket to be still connected. Thus it sends the new request using this stale connection and hence we get EOFException.

The best way to handle this is to check the Response Headers after each request you send. The server always sends a "Connection: Close" before terminating a connection (HTTP 1.1). So, you can use getHeaderField() and check for "Connection" field. Another thing to note is that server ONLY sends this connection field when it is about to terminate the connection. So, you need to code around this with the possibility of getting a "null" in the normal case (when server is not closing the connection)

Thank you for explaining what's going on, rather than just giving an unexplained command for someone to execute! Much appreciated. [+1] — naki, Feb 25 '16 at 04:59

James Wald · Answer 3 · 2014-05-22T19:26:11.030

This workaround tends to be reliable and performant:

static final int MAX_CONNECTIONS = 5;

T send(..., int failures) throws IOException {
    HttpURLConnection connection = null;

    try {
        // initialize connection...

        if (failures > 0 && failures <= MAX_CONNECTIONS) {
            connection.setRequestProperty("Connection", "close");
        }

        // return response (T) from connection...
    } catch (EOFException e) {
        if (failures <= MAX_CONNECTIONS) {
            disconnect(connection);
            connection = null;

            return send(..., failures + 1);
        }

        throw e;
    } finally {
        disconnect(connection);
    }
}

void disconnect(HttpURLConnection connection) {
    if (connection != null) {
        connection.disconnect();
    }
}

This implementation relies on the fact that the default number of connections that can be opened with a server is 5 (Froyo - KitKat). This means that up to 5 stale connections may exist, each of which will have to be closed.

After each failed attempt, the Connection:close request property will cause the underlying HTTP engine to close the socket when connection.disconnect() is called. By retrying up to 6 times (max connections + 1), we ensure that the last attempt will always be given a new socket.

The request may experience additional latency if no connections are alive, but that is certainly better than an EOFException. In that case, the final send attempt won't immediately close the freshly opened connection. That's the only practical optimization that can be made.

Instead of relying on the magic default value of 5, you may be able to configure the system property yourself. Keep in mind that this property is accessed by a static initializer block in KitKat's ConnectionPool.java, and it works like this in older Android versions too. As a result, the property may be used before you have a chance to set it.

static final int MAX_CONNECTIONS = 5;

static {
    System.setProperty("http.maxConnections", String.valueOf(MAX_CONNECTIONS));
}

score 4 · Answer 4 · answered Apr 29 '16 at 03:18

Yes. There is a problem in the Android platform, specifically, in Android libcore with version 4.1-4.3.

The problem is introduced in this commit: https://android.googlesource.com/platform/libcore/+/b2b02ac6cd42a69463fd172531aa1f9b9bb887a8

Android 4.4 switched http lib to "okhttp" which doesn't have this problem.

Problem explained as follow:

On android 4.1-4.3, when you are using URLConnection/HttpURLConnection to POST with "ChunkedStreamingMode" or "FixedLengthStreamingMode" set, URLConnection/HttpURLConnection will not do silent retry if the reused connection is stale. You should retry POST at most "http.maxConnections+1" times in your code, just as previous answers suggest.

score 1 · Answer 5 · answered Jan 08 '15 at 17:20

I suspect it might be the server that is at fault here, and the HttpURLConnection is not as forgiving as other implementations. That was the cause of my EOFException. I suspect in my case this would not be intermittent (fixed it before testing the N retry workaround), so the answers above relate to other issues and be a correct solution in those cases.

My server was using python SimpleHTTPServer and I was wrongly assuming all I needed to do to indicate success was the following:

self.send_response(200)

That sends the initial response header line, a server and a date header, but leaves the stream in the state where you are able to send additional headers too. HTTP requires an additional new line after headers to indicate they are finished. It appears if this new line isn't present when you attempt to get the result body InputStream or response code etc with HttpURLConnection then it throws the EOFException (which is actually reasonable, thinking about it). Some HTTP clients did accept the short response and reported the success result code which lead to me perhaps unfairly pointing the finger at HttpURLConnection.

I changed my server to do this instead:

self.send_response(200)
self.send_header("Content-Length", "0")
self.end_headers()

No more EOFException with that code.

score 0 · Answer 6 · answered Dec 15 '15 at 10:19

This worked for me.

public ResponseObject sendPOST(String urlPrefix, JSONObject payload) throws JSONException {
    String line;
    StringBuffer jsonString = new StringBuffer();
    ResponseObject response = new ResponseObject();
    try {

        URL url = new URL(POST_URL);
        HttpURLConnection connection = (HttpURLConnection) url.openConnection();
        connection.setDoInput(true);
        connection.setDoOutput(true);
        connection.setReadTimeout(10000);
        connection.setConnectTimeout(15000);
        connection.setRequestMethod("POST");
        connection.setRequestProperty("Accept", "application/json");
        connection.setRequestProperty("Content-Type", "application/json; charset=UTF-8");

        OutputStream os = connection.getOutputStream();
        os.write(payload.toString().getBytes("UTF-8"));
        os.close();
        BufferedReader br = new BufferedReader(new InputStreamReader(connection.getInputStream()));
        while ((line = br.readLine()) != null) {
            jsonString.append(line);
        }
        response.setResponseMessage(connection.getResponseMessage());
        response.setResponseReturnCode(connection.getResponseCode());
        br.close();
        connection.disconnect();
    } catch (Exception e) {
        Log.w("Exception ",e);
        return response;
    }
    String json = jsonString.toString();
    response.setResponseJsonString(json);
    return response;
}

Nizzy · Answer 7 · 2014-06-30T10:36:17.423

-5

connection.addRequestProperty("Accept-Encoding", "gzip");

is the answer

edited Jun 30 '14 at 10:36

answered Jun 30 '14 at 06:48

Nizzy

1,879
3
21
33

Android HttpUrlConnection EOFException

7 Answers7

Linked