10

I am using HttpClient 4.0.1 on android... I make a POST request with a header set that is the current millis... I see that request hit the server twice within a few millis (5-10) of each other.. but the header I set is the same for both requests. This happens very sporadically... I see no real difference between the requests in wireshark... I just have no clue how this could be happening. Anyone run into this before or have any tips on how to further debug it?

here is the code I use to create the client:

public static HttpClient getAndroidHttpClient(final int timeOut) {
    // set up the schemas
    SchemeRegistry schemeRegistry = new SchemeRegistry();
    schemeRegistry.register(new Scheme("http", PlainSocketFactory.getSocketFactory(), 80));
    schemeRegistry.register(new Scheme("https", new EasySSLSocketFactory(), 443));

    // set up our params
    HttpParams params = new BasicHttpParams();
    params.setIntParameter(CoreConnectionPNames.CONNECTION_TIMEOUT, timeOut);
    params.setIntParameter(CoreConnectionPNames.STALE_CONNECTION_CHECK, timeOut);
    params.setIntParameter(CoreConnectionPNames.SO_TIMEOUT, timeOut);
    params.setLongParameter(ConnManagerPNames.TIMEOUT, timeOut);
    params.setParameter(ConnManagerPNames.MAX_TOTAL_CONNECTIONS, 1);
    params.setParameter(ConnManagerPNames.MAX_CONNECTIONS_PER_ROUTE, new ConnPerRouteBean(1));
    params.setParameter(HttpProtocolParams.USE_EXPECT_CONTINUE, false);

    HttpProtocolParams.setUserAgent(params, "android-client-v1.0");
    HttpProtocolParams.setVersion(params, HttpVersion.HTTP_1_1);
    HttpProtocolParams.setContentCharset(params, "utf8");

    ThreadSafeClientConnManager conman = new ThreadSafeClientConnManager(params, schemeRegistry);

    DefaultHttpClient defaultHttpClient = new DefaultHttpClient(conman, params);

    return defaultHttpClient;
}
danb
  • 10,239
  • 14
  • 60
  • 76
  • 2
    If you disable retries (i.e. `DefaultHttpClient.setHttpRequestRetryHandler(new DefaultHttpRequestRetryHandler(0, false))`), does this still occur? – Marvin Pinto Jan 23 '12 at 16:06
  • Maybe it comes from the SchemeRegistry ? Maybe take a look at http://stackoverflow.com/a/1343130/320180 – ndeverge Jan 23 '12 at 16:05
  • Marvin, this seems to be the problem... if you want the bounty, add an answer. Unfortunately, now I'm running into this issue: http://stackoverflow.com/questions/2052299/httpclient-on-android-nohttpresponseexception-through-umts-3g which is problematic because sending this same request again causes weird behavior on the server, which is the issue I was trying to solve in the first place... – danb Jan 23 '12 at 16:37
  • 1
    @danb Will do - give me a bit to do a bit more research and write it up. – Marvin Pinto Jan 23 '12 at 16:38
  • @danb I saw your comment on the [other](http://stackoverflow.com/a/5043412/1101070) post re: "the server gets the first request and changes some state.. sending that same request again really borks things". Is it feasible at this point to tweak how the server handles duplicate requests? I'm thinking: when it receives a duplicate request (i.e. timestamp defined duplicate), respond with how you would have _originally_ responded. Could you perhaps provide an overview of how this works so I can get a better idea? – Marvin Pinto Jan 23 '12 at 17:11
  • part of the problem is that this bug causes the dup POST request so quickly that the first request hasn't even completed processing yet. From the server's perspective there are two "simultaneous" authentication requests happening. Which in and of itself isn't bad, but our session creation can trigger business logic which is also duplicated for this particular user. We can certainly hack around it.. but it won't be pretty. – danb Jan 23 '12 at 23:04

3 Answers3

20

So what appears to be happening here is that your client sends a request, does not get a response in a timely manner, and as a result retries the same request again (as it should). This, in turn, results in multiple POST requests being sent to your server (almost in succession), which your server cannot currently deal with appropriately.

To verify/debug this, try disabling HTTP retries as follows:

defaultHttpClient.setHttpRequestRetryHandler(new DefaultHttpRequestRetryHandler
                                             (0, false));

This of course will deal with your duplicate requests issue, but then introduces another more serious issue; namely, it will try once (and only once) and fail. From the information I got from your comments, here are a few ideas you can try:

Please bare with the pseudocode as I don't have all the details of how your client is architected

Handle Multiple POSTS in Succession

  • Disable automatic retries (as above)
  • Wrap your POST requests in a loop similar to how this is implemented
  • Then either sleep between your manual retries or implement your version of exponential backoff

No matter what, your server will need the capability to handle duplicate requests in a reasonable way, this is HTTP afterall. However, you're at least giving it a chance to process the first one before it's bombarded with duplicates.

I recommend that the first step it takes when processing a request is to set some form of a (duplicate) flag. Then if/when it receives a dupe, it continues processing the first request (as usual) and silently ignores the dupes.

So just to summarize, the point of this whole scheme was to give your server a chance to set the dupe flag. After that, it's your server's job to discard (or handle) duplicate requests as needed. Does this all make sense?

Marvin Pinto
  • 30,138
  • 7
  • 37
  • 54
  • 1
    Thanks Marvin, your suggestion lead us down the right path.. now at least we know what we're dealing with. – danb Jan 24 '12 at 17:48
1

I cannot speak for HttpClient version shipped with Android, as it is effectively a fork now based on an extremely old pre-BETA snapshot. However, if you are using a stock version of Apache HttpClient 4.x, it DOES NOT automatically retry POST or PUT requests unless configured to do otherwise.

In your particular case I suspect the HTTP message gets retransmitted by the wireless driver due to loss of connectivity or similar networking issue. HTTP is not a guaranteed delivery protocol. HTTP messages can get re-sent by lower level transports. Your application must be prepared to deal with duplicate HTTP messages.

ok2c
  • 26,450
  • 5
  • 63
  • 71
  • I thought I ran into a old bug report that said there was a problem somewhere in 4.0 with retrys affecting posts and puts.. but now I can't find it... but Marvins solution did eliminate the second request.. so now I just get "no response" exceptions instead of the dup. – danb Jan 24 '12 at 17:47
  • HTTP is not really a delivery protocol at all, it is an application protocol. HTTP is transmitted over TCP, which is the delivery layer. TCP is defined to be a reliable delivery protocol that handles retransmissions and provides the guarantee. I assume that's why @oleg's answer was down-voted. – Suncat2000 Feb 13 '14 at 21:56
  • @Suncat2000: by non-guaranteed delivery I mean that HTTP protocol makes no provisions for message retransmission in case of a transport failure. Say, the server successfully receives and processes the request but the response is never delivered to the client. HTTP defines some methods to be non-idempotent instead and assumes that both endpoints have a common strategy for dealing with those. – ok2c Feb 15 '14 at 14:20
0

Retry PolicyWe can customized

WITH VOLLEY REQUEST

stringRequest.setRetryPolicy(new DefaultRetryPolicy( 0, DefaultRetryPolicy.DEFAULT_MAX_RETRIES, DefaultRetryPolicy.DEFAULT_BACKOFF_MULT)); volleySingleton.addToRequestQueue(stringRequest);

Akash
  • 1
  • 3