I am having a problem in my Spring Boot application where I am using a RestTemplate which is autowired for making POST calls to a REST service. The code where I am initializing the REstTemplate bean is as follows:
@Bean(name = "template")
public RestTemplate restTemplate(RestTemplateBuilder builder) throws NoSuchAlgorithmException, KeyManagementException {
TrustManager[] trustAllCerts = new TrustManager[] {
new X509TrustManager() {
public java.security.cert.X509Certificate[] getAcceptedIssuers() {
return new X509Certificate[0];
}
public void checkClientTrusted(
java.security.cert.X509Certificate[] certs, String authType) {
}
public void checkServerTrusted(
java.security.cert.X509Certificate[] certs, String authType) {
}
}
};
SSLContext sslContext = SSLContext.getInstance("SSL");
sslContext.init(null, trustAllCerts, new java.security.SecureRandom());
HostnameVerifier hostnameVerifier = SSLConnectionSocketFactory.getDefaultHostnameVerifier();
// Conn Pool
PoolingHttpClientConnectionManager cm = new PoolingHttpClientConnectionManager(
RegistryBuilder.<ConnectionSocketFactory>create()
.register("http", PlainConnectionSocketFactory.getSocketFactory())
.register("https", new SSLConnectionSocketFactory(sslContext, hostnameVerifier))
.build());
cm.setMaxTotal(Integer.MAX_VALUE);
cm.setDefaultMaxPerRoute(50);
CloseableHttpClient httpClient = HttpClients.custom()
.setSSLContext(sslContext)
.setConnectionManager(cm)
.setSSLHostnameVerifier(NoopHostnameVerifier.INSTANCE)
.build();
HttpComponentsClientHttpRequestFactory customRequestFactory = new HttpComponentsClientHttpRequestFactory();
customRequestFactory.setHttpClient(httpClient);
return builder.requestFactory(() -> customRequestFactory).build();
}
The RestTemplate is then autowired:
@Autowired
@Qualifier("template")
private RestTemplate restTemplate;
And the RestTemplate is used thus:
restResponse = restTemplate.postForObject(targetUrl, obj, CustomRestResponse.class);
I am deploying the application to PCF where it is running with 6 instances. It's then giving me an error like this:
org.springframework.web.client.RestClientException: Error while extracting response for type [class com.test.model.CustomRestResponse] and content type [application/json;charset=UTF-8]; nested exception is org.apache.http.ConnectionClosedException: Premature end of chunk coded message body: closing chunk expected
To replicate the issue in my local environment, I created a multithreaded application which will spawn multiple threads and call the REST service simultaneously:
ExecutorService service = Executors.newFixedThreadPool(10);
IntStream.range(0, 100).forEach(i -> service.submit(new RestTemplateTesterRunnable(i)));
And I was able to get the exception in my local system as well (Note that I don't have this multithreading code in the actual application. This is just for replicating the problem in my local since I can't run 6 instances automatically). The full stacktrace is as follows:
org.springframework.web.client.RestClientException: Error while extracting response for type [class com.test.model.CustomRestResponse] and content type [application/json;charset=UTF-8]; nested exception is org.apache.http.ConnectionClosedException: Premature end of chunk coded message body: closing chunk expected
at org.springframework.web.client.HttpMessageConverterExtractor.extractData(HttpMessageConverterExtractor.java:119)
at org.springframework.web.client.RestTemplate.doExecute(RestTemplate.java:744)
at org.springframework.web.client.RestTemplate.execute(RestTemplate.java:677)
at org.springframework.web.client.RestTemplate.postForObject(RestTemplate.java:421)
at com.test.RestTemplateTester$RestTemplateTesterRunnable.run(RestTemplateTester.java:237)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Caused by: org.apache.http.ConnectionClosedException: Premature end of chunk coded message body: closing chunk expected
at org.apache.http.impl.io.ChunkedInputStream.getChunkSize(ChunkedInputStream.java:263)
at org.apache.http.impl.io.ChunkedInputStream.nextChunk(ChunkedInputStream.java:222)
at org.apache.http.impl.io.ChunkedInputStream.read(ChunkedInputStream.java:183)
at org.apache.http.impl.io.ChunkedInputStream.read(ChunkedInputStream.java:210)
at org.apache.http.impl.io.ChunkedInputStream.close(ChunkedInputStream.java:312)
at org.apache.http.impl.execchain.ResponseEntityProxy.streamClosed(ResponseEntityProxy.java:142)
at org.apache.http.conn.EofSensorInputStream.checkClose(EofSensorInputStream.java:228)
at org.apache.http.conn.EofSensorInputStream.close(EofSensorInputStream.java:172)
at java.io.PushbackInputStream.close(PushbackInputStream.java:379)
at com.fasterxml.jackson.core.json.UTF8StreamJsonParser._closeInput(UTF8StreamJsonParser.java:252)
at com.fasterxml.jackson.core.base.ParserBase.close(ParserBase.java:369)
at com.fasterxml.jackson.databind.ObjectMapper._readMapAndClose(ObjectMapper.java:4210)
at com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:3258)
at org.springframework.http.converter.json.AbstractJackson2HttpMessageConverter.readJavaType(AbstractJackson2HttpMessageConverter.java:239)
at org.springframework.http.converter.json.AbstractJackson2HttpMessageConverter.read(AbstractJackson2HttpMessageConverter.java:227)
at org.springframework.web.client.HttpMessageConverterExtractor.extractData(HttpMessageConverterExtractor.java:104)
... 9 more
However, I am not being able to solve the problem. I referred this question:
How to efficiently use RestTemplate in multithreading application?
and this one:
How to use RestTemplate efficiently in Multithreaded environment?
and in both of them, people are saying that RestTemplate is thread-safe and there shouldn't be a problem in using it in a multithreaded environment. However, what I am seeing is that this exception is still happening for me. I thought at first that this was happening due to my use of the CloseableHttpClient, as is given by this post:
ConnectionClosedException: Premature end of chunk coded message body with apache hhtpclient 4.5.5
However, as I am not directly using the client, which is internally used by Spring Boot, I didn't have much to do about it. But I looked into the internal working of the RestTemplate's postForObject() method in any case, which led me to this method:
@Nullable
protected <T> T doExecute(URI url, @Nullable HttpMethod method, @Nullable RequestCallback requestCallback,
@Nullable ResponseExtractor<T> responseExtractor) throws RestClientException {
Assert.notNull(url, "URI is required");
Assert.notNull(method, "HttpMethod is required");
ClientHttpResponse response = null;
try {
ClientHttpRequest request = createRequest(url, method);
if (requestCallback != null) {
requestCallback.doWithRequest(request);
}
response = request.execute();
handleResponse(url, method, response);
return (responseExtractor != null ? responseExtractor.extractData(response) : null);
}
catch (IOException ex) {
String resource = url.toString();
String query = url.getRawQuery();
resource = (query != null ? resource.substring(0, resource.indexOf('?')) : resource);
throw new ResourceAccessException("I/O error on " + method.name() +
" request for \"" + resource + "\": " + ex.getMessage(), ex);
}
finally {
if (response != null) {
response.close();
}
}
}
I am seeing that before returning, the response object is being closed, as was being told in the previous StackOverflow post I linked. Can this be the reason for the ConnectionClosedException in a multithreaded environment? If so, what is the solution? I tried to use evictIdleConnections() as given by the Javadocs:
So the code became:
CloseableHttpClient httpClient = HttpClients.custom()
.setSSLContext(sslContext)
.setConnectionManager(cm)
.setSSLHostnameVerifier(NoopHostnameVerifier.INSTANCE)
.evictIdleConnections(10, TimeUnit.MILLISECONDS)
.build();
However, this didn't help. I also saw that some people are using org.apache.commons.httpclient.MultiThreadedHttpConnectionManager, but I found that this is a deprecated connection manager, and the HttpComponentsClientHttpRequestFactory does not support it. Nevertheless, I somehow used it also, but with no results. Please hellp me in getting the actual reason for this exception, and how to resolve it.
