Connection reset on web service with Apache camel

Question

I have apache camel based integration engine deployed in apache-tomcat which uses common-http client for http call to external webservices. Most of the cases getting successful call, but rarely or the first call after server restart, I'm getting Connection Reset Error.

This is not related to socket timeout or connection timeout which should be applicable for all time. I have enabled http level logging and trace given below.

(Component versions camel-http-2.18.2.jar, commons-httpclient-3.1.jar, apache-tomcat-8.5.11)

apache.camel.component.http.HttpProducer - Executing http POST method:
org.apache.commons.httpclient.MultiThreadedHttpConnectionManager - HttpConnectionManager.getConnection:  
config = HostConfiguration[host=https://nodeD1.test.webservices.test.com], timeout = 0
org.apache.commons.httpclient.MultiThreadedHttpConnectionManager - Getting free connection, 
hostConfig=HostConfiguration[host=https://nodeD1.test.webservices.test.com]
httpclient.wire.header - >> "POST /1ASIWPLRHUR HTTP/1.1[\r][\n]"
httpclient.wire.header - >> "http-url:[https://nodeD1.test.webservices.test.com]"
httpclient.wire.header - >> "httpClient.soTimeout: 30000[\r][\n]"
httpclient.wire.header - >> "SOAPAction: [edited]"
httpclient.wire.header - >> "transport-type: http[\r][\n]"
httpclient.wire.header - >> "x-sequence-id: 1.101.102.101[\r][\n]"
httpclient.wire.header - >> "User-Agent: Jakarta Commons-HttpClient/3.1[\r][\n]"
httpclient.wire.header - >> "Host: nodeD1.test.webservices.test.com[\r][\n]"
httpclient.wire.header - >> "Content-Length: 4505[\r][\n]"
httpclient.wire.header - >> "Content-Type: text/xml,charset=UTF-8[\r][\n]"
httpclient.wire.header - >> "[\r][\n]"
httpclient.wire.content - >> "<?xml version="1.0" encoding="UTF-8"?>[\n]"
httpclient.wire.content - >> "[edited]"
apache.commons.httpclient.methods.EntityEnclosingMethod - Request body sent
org.apache.commons.httpclient.HttpMethodDirector - Closing the connection.
org.apache.commons.httpclient.HttpMethodDirector - Method retry handler returned false. Automatic recovery will not be attempted
org.apache.commons.httpclient.HttpConnection - Releasing connection back to connection manager.
org.apache.commons.httpclient.MultiThreadedHttpConnectionManager - Freeing connection, 
hostConfig=HostConfiguration[host=https://nodeD1.test.webservices.test.com]
apache.commons.httpclient.util.IdleConnectionHandler - Adding connection at: 1547029324609
org.apache.commons.httpclient.MultiThreadedHttpConnectionManager - Notifying no-one, there are no waiting threads
org.apache.camel.processor.DefaultErrorHandler - Failed delivery on delivery attempt: 0 caught: java.net.SocketException: Connection reset
ERROR org.apache.camel.processor.DefaultErrorHandler - Failed delivery for 
Exhausted after delivery attempt: 1 caught: java.net.SocketException: Connection reset

java.net.SocketException: Connection reset
        at java.net.SocketInputStream.read(SocketInputStream.java:210)
        at java.net.SocketInputStream.read(SocketInputStream.java:141)
        at sun.security.ssl.InputRecord.readFully(InputRecord.java:465)
        at sun.security.ssl.InputRecord.read(InputRecord.java:503)
        at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:983)
        at sun.security.ssl.SSLSocketImpl.readDataRecord(SSLSocketImpl.java:940)
        at sun.security.ssl.AppInputStream.read(AppInputStream.java:105)
        at java.io.BufferedInputStream.fill(BufferedInputStream.java:246)
        at java.io.BufferedInputStream.read(BufferedInputStream.java:265)
        at org.apache.commons.httpclient.HttpParser.readRawLine(HttpParser.java:78)
        at org.apache.commons.httpclient.HttpParser.readLine(HttpParser.java:106)
        at org.apache.commons.httpclient.HttpConnection.readLine(HttpConnection.java:1116)
        at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.readLine(MultiThreadedHttpConnectionManager.java:1413)
        at org.apache.commons.httpclient.HttpMethodBase.readStatusLine(HttpMethodBase.java:1973)
        at org.apache.commons.httpclient.HttpMethodBase.readResponse(HttpMethodBase.java:1735)
        at org.apache.commons.httpclient.HttpMethodBase.execute(HttpMethodBase.java:1098)
        at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:398)
        at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171)
        at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397)
        at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323)
        at org.apache.camel.component.http.HttpProducer.executeMethod(HttpProducer.java:257)
        at org.apache.camel.component.http.HttpProducer.process(HttpProducer.java:152)
        at org.apache.camel.util.AsyncProcessorConverterHelper$ProcessorToAsyncProcessorBridge.process(AsyncProcessorConverterHelper.java:61)
        at org.apache.camel.processor.SendDynamicProcessor$1.doInAsyncProducer(SendDynamicProcessor.java:124)
        at org.apache.camel.impl.ProducerCache.doInAsyncProducer(ProducerCache.java:436)
        at org.apache.camel.processor.SendDynamicProcessor.process(SendDynamicProcessor.java:119)

Can someone help me find out what actually causing this connection reset issue intermittent?

Quoting this [answer](https://stackoverflow.com/a/63155/1377895) `Connection reset simply means that a TCP RST was received. This happens when your peer receives data that it can't process, and there can be various reasons for that.` Apaches HttpClient 4.4+ i.e. [reuses connection but only checks every 2 seconds if the connection is stale (half-closed or closed)](http://hc.apache.org/httpcomponents-client-ga/httpclient/apidocs/org/apache/http/impl/conn/PoolingHttpClientConnectionManager.html). In such a case it might attempt to reuse a closed connectionand receive a RST package in response — Roman Vottner, Feb 04 '19 at 16:48
Not sure about commons-http as it got officially [replaced by Apache's HttpClient](http://hc.apache.org/httpclient-3.x/) — Roman Vottner, Feb 04 '19 at 16:49
@RomanVottner yeah common-http replaced by HttpClient, but didn't get any bug info related connection reset, surprisingly this is only repeatable in aws cloud environment and none of any non-cloud internal environment. — Praveesh, Feb 04 '19 at 20:48

Connection reset on web service with Apache camel

0 Answers0