HTTPClient hung or slow in multi-threaded environment

Question

I am using following version of httpclient api to call the rest webservice. httpcore4.4.9 httpclient 4.5.9 I am using the ExecutorService to execute the task where it is calling my following methods to get the data from rest API. ExecutorService execute about 30+ rest call in one transaction and i observed that, httpclient HUNG or SLOW for few of the rest call and it hit the performance issue. Can you please check if below i am using httpclient correct way ?

        String output = null;
        CloseableHttpClient httpClient = getHttpClients();

        try (CloseableHttpResponse response = httpClient.execute(httpGet)) {
            HttpEntity entity = response.getEntity();
            output = EntityUtils.toString(entity);
            EntityUtils.consume(response.getEntity());
        } catch (Exception e) {
            log.error("error during call to REST API " + httpGet.getURI() + "   ", e);
        } finally {
            httpClient.close();
            httpGet.releaseConnection();
            httpGet.abort();
        }
        return output;
    }

    private CloseableHttpClient getHttpClients() {
        long startTime = System.currentTimeMillis();
        long endTime = 0;
        int timeout = 12000;
        RequestConfig.Builder requestBuilder = RequestConfig.custom();
        requestBuilder.setConnectTimeout(timeout);
        requestBuilder.setConnectionRequestTimeout(timeout);

        SocketConfig socketConfig = SocketConfig.custom().setSoTimeout(timeout).build();

        HttpClientBuilder builder = HttpClientBuilder.create();
        builder.setDefaultRequestConfig(requestBuilder.build());
        builder.disableContentCompression();
        builder.setDefaultSocketConfig(socketConfig);
        CloseableHttpClient httpClient = builder
                .build();
        endTime = System.currentTimeMillis();
        log.debug("Total time took to build client = " + (endTime - startTime));
        return httpClient;
    } ```

score 0 · Answer 1 · answered Jul 30 '20 at 04:53

As per my experience, this problem is occurring because of EntityUtils.consume(response.getEntity()) .
You could find what it actually does here.

Try removing and running your code to match the speed and decide if you want it or not.
Below is the code used by me:

CloseableHttpClient httpClient = HttpClients.createDefault();
    try {
      HttpGet request = new HttpGet(url);
      CloseableHttpResponse response = httpClient.execute(request);
      String responseString = EntityUtils.toString(response.getEntity());
} catch (Exception ex) {
      log.error(ex.toString());
    }

score 0 · Answer 2 · answered Aug 27 '20 at 06:56

0

Try to adjust MaxTotal and MaxPerRoute variable in PoolingHttpClientConnectionManager class.

connectionManager.setMaxTotal(maxTotalConnections);
connectionManager.setDefaultMaxPerRoute(maxConnectionsPerRoute);

answered Aug 27 '20 at 06:56

Juey

123
1
6

HTTPClient hung or slow in multi-threaded environment

2 Answers2