HttpClient hangs on socketRead0 with successfully executed method

Question

In our web-application A user can submit a url. We will fetch the data and parse it server side. For each request we use an HttpClient with the following (relevant) settings

connectionManager.getParams().setConnectionTimeout(10000);
connectionManager.getParams().setSoTimeout(10000);

When I call HttpMethod.getResponseBody the status code has already been checked to be acceptable. At this point the thread hangs with this stack trace:

java.net.SocketInputStream.socketRead0 ( native code )
java.net.SocketInputStream.read ( SocketInputStream.java:150 )
java.net.SocketInputStream.read ( SocketInputStream.java:121 )
java.io.BufferedInputStream.read1 ( BufferedInputStream.java:273 )
java.io.BufferedInputStream.read ( BufferedInputStream.java:334 )
java.io.FilterInputStream.read ( FilterInputStream.java:133 )
org.apache.commons.httpclient.AutoCloseInputStream.read ( AutoCloseInputStream.java:108 )
java.io.FilterInputStream.read ( FilterInputStream.java:107 )
org.apache.commons.httpclient.AutoCloseInputStream.read ( AutoCloseInputStream.java:127 )
org.apache.commons.httpclient.HttpMethodBase.getResponseBody ( HttpMethodBase.java:690 )

I cannot discover the exact URL for which this happened (was an incident on a live environment) and I have been unable to reproduce it. I'd like to think it's simply a matter of the server we're connecting to behaving strangely, but perhaps I'm missing something. In either case, is there a way for me to prevent the blocking method call from waiting forever? The SoTimeout is also the socket read timeout? Is there another setting I'm missing?

score 4 · Answer 1 · answered Oct 04 '13 at 19:13

4

I have all the timeouts setup just fine but I found out we have on url that does http chunking but sends no results(works fine in chrome, but in http client it hangs forever even with the timeout set). Luckily I own the server and just return some garbage and it no longer hangs. This seems like a very unique bug in that http client does not handle some kind of empty chunking case well(though I could be way off)....I just know it hangs every time on that same url with empty data and that url is http chunking csv download back to our http client.

answered Oct 04 '13 at 19:13

Dean Hiller

19,235
25
129
212

@Dean Hiller: this makes no sense. Socket timeout applies to socket read operations and in no way related to any HTTP protocol elements. If you can still reproduce this issue please raise a JIRA with HC project and submit a wire log of the session exhibiting the problem. – ok2c Apr 16 '14 at 12:57
I agree with you and in my experience the timeout usually works...could have been something else I screwed up but returning data fixed the issue. – Dean Hiller Apr 16 '14 at 23:38

score 1 · Answer 2 · answered Jan 04 '16 at 18:59

HttpClient distinguishes between connection and request. setSoTimeout will configure the connection socket timeout while setConnectionTimeout will configure both the timeout for the connection manager (how long to wait for a connection) and for the establishment of the connection itself. In the code you provided, you are not setting any timeout for the socket used for the request itself, and unfortunately, HttpClient has no timeout by default for that.

Here's how I do it in v4.4.1:

// Configure the socket timeout for the connection, incl. ssl tunneling
connManager = new PoolingHttpClientConnectionManager();
connManager.setMaxTotal(200);
connManager.setDefaultMaxPerRoute(100);

SocketConfig sc = SocketConfig.custom()
    .setSoTimeout(soTimeoutMs)
    .build();

connManager.setDefaultSocketConfig(sc);

HttpClient client = HttpClients.custom()
            .setConnectionManager(connManager)
            .setConnectionManagerShared(true)
            .build();

// configure the timeouts (socket and connection) for the request
RequestConfig.Builder config = = RequestConfig.copy(RequestConfig.DEFAULT);
config.setConnectionRequestTimeout(connectionTimeoutMs);
config.setSocketTimeout(socketTimeoutMs);

HttpRequestBase req = new HttpGet(uri);
req.setConfig(config.build());

client.execute(req);

`setConnectTimeout` != `setConnectionRequestTimeout` and there is no `setConnectionTimeout ` — Antoniossss, Aug 30 '16 at 08:21

score 0 · Answer 3 · answered Apr 08 '13 at 08:59

0

When I call HttpMethod.getResponseBody the status code has already been checked to be acceptable. At this point the thread hangs

Looks like you have a problem with synchronizing the calls ... you should ensure that the method

HttpMethod.getResponseBody

is called sequentially or should use a mutex (semaphore ) for the part that changes the status code

Also you should decrease your timeout limit to prevent hangs.

answered Apr 08 '13 at 08:59

Stephan

8,000
3
36
42

Checking if the method succeeded and getting the response happens on the same thread. There is no concurrency issue. HttpClient fetches and updates the status. The call hangs indefinitely, the mentione timeout is not honored, changing it has no effect. – Kafkaesque Apr 09 '13 at 08:52
mine is worse and happens every single time in a threaded application though it takes 800 calls or so before it happens so it's a real pain to debug....incidentally it hangs on a call that typically chunks data back(http chunking) but this one call is where there is no data yet for some reason http client hangs....could be server doing something wrong though I try the url manually in chrome and it works fine...only http client is struggling with it. – Dean Hiller Oct 04 '13 at 18:49
I posted our stack trace which is very close – Dean Hiller Oct 04 '13 at 18:54

score 0 · Answer 4 · answered Oct 04 '13 at 18:54

We see this consistently in our implementation and it looks like http client is not handling bad servers correctly or something and it is not timing out.....I can reproduce in our setting with this databus open source project and the stack trace is a little different...

SocketInputStream.socketRead0(FileDescriptor, byte[], int, int, int) line: not available [native method]
SocketInputStream.read(byte[], int, int) line: 129
SocketInputBuffer(AbstractSessionInputBuffer).fillBuffer() line: 166
SocketInputBuffer.fillBuffer() line: 90 SocketInputBuffer(AbstractSessionInputBuffer).readLine(CharArrayBuffer) line: 281
DefaultHttpResponseParser.parseHead(SessionInputBuffer) line: 92
DefaultHttpResponseParser.parseHead(SessionInputBuffer) line: 62
DefaultHttpResponseParser(AbstractMessageParser).parse() line: 254
DefaultClientConnection(AbstractHttpClientConnection).receiveResponseHeader() line: 289 DefaultClientConnection.receiveResponseHeader() line: 252
BasicPooledConnAdapter(AbstractClientConnAdapter).receiveResponseHeader() line: 219 HttpRequestExecutor.doReceiveResponse(HttpRequest, HttpClientConnection, HttpContext) line: 300 HttpRequestExecutor.execute(HttpRequest, HttpClientConnection, HttpContext) line: 127
DefaultRequestDirector.tryExecute(RoutedRequest, HttpContext) line: 712 DefaultRequestDirector.execute(HttpHost, HttpRequest, HttpContext) line: 517
DefaultHttpClient(AbstractHttpClient).execute(HttpHost, HttpRequest, HttpContext) line: 906 DefaultHttpClient(AbstractHttpClient).execute(HttpUriRequest, HttpContext) line: 805
ReadAggregations.processAggregation(String, Counter, Counter, Counter, Counter) line: 153
ReadAggregations.start() line: 96
ReadAggregations.main(String[]) line: 70

I'm still facing this issue today. Do you know any alternatives to apache httpclient that does not hang with bad servers? — Arya, Sep 07 '18 at 08:29
@Arya I ended up writing an http1.1 client and an http2 client here https://github.com/deanhiller/webpieces (this is a WHOLE slew of webpieces). This is very beta though BUT has a unique backpressure feature (that for some reason needs to be turned of right now in the client....as I am working on that bug). ps. it looks like a webserver only but the webserver is built on the same http parser, same nio layer, etc that the client is built on and the client is asynchronous. while it is beta, it is written for testing like — Dean Hiller, Sep 07 '18 at 13:55
@Arya to finish that sentence, like this https://blog.twitter.com/engineering/en_us/topics/insights/2017/the-testing-renaissance.html NOTE: The webserver is way more tested than the client(which ends up testing the http parsers), but I happen to be working on the client recently. — Dean Hiller, Sep 07 '18 at 13:56
@Arya lol, holy crap, I totally forgot, there is also an http2to1_1-client meaning you deal with http2 objects and talk over http1.1 protocol such that it swaps 1 to 1 with the http2 client. (ie. the apis are the same). I forgot I did that. lol. I have been documenting the webserver but not the client as of yet :( so I still need to do that though there is not much to it as the objects are 1 to 1 with the protocol leaving the client to deal with just objects that come in and go out. — Dean Hiller, Sep 07 '18 at 14:00

score 0 · Answer 5 · edited May 23 '17 at 11:46

0

You can try to abort the request with HttpUriRequest#abort(), see https://hc.apache.org/httpcomponents-client-4.3.x/httpclient/apidocs/org/apache/http/client/methods/HttpUriRequest.html#abort%28%29. However, setting a timemout that requires no intercepting would be nicer. Here is a related question: Setting time out in apache http client

edited May 23 '17 at 11:46

Community

1
1

answered Apr 24 '14 at 09:38

static-max

739
10
19

HttpClient hangs on socketRead0 with successfully executed method

5 Answers5

Linked