How to force Apache HttpClient / HttpConnection to absolutely close tcp connection?

Question

I use lot of http connection with Apache HttpClient (org.apache.httpcomponents 4.3.6) to test servers and I cannot force connections to close, even when I schedule HttpConnectionManager to httpClient.getConnectionManager().shutdown(); after 10 seconds in another thread.

httpClient.close() also doesn't help.

Connections can last for minutes or even hours.

I have tried custom SocketConfig and this also not helps:

 SocketConfig socketConfig = SocketConfig.custom()
                .setSoKeepAlive(false)
                .setSoLinger(5)
                .setSoReuseAddress(true)
                .setSoTimeout(5000)
                .setTcpNoDelay(true).build();

The way I am fetching the content:

  HttpUriRequest request = new HttpGet(url);
  HttpResponse response = httpClient.execute(request);

  try (InputStream in = response.getEntity().getContent()) {
       String result = IOUtils.toString(in, StandardCharsets.UTF_8);
       httpClient.close();
       return result;
  }

The way I am building HTTP Client:

    SocketConfig socketConfig = SocketConfig.custom()
            .setSoKeepAlive(false)
            .setSoLinger(configuration.getInt("proxy.list.socket.so.linger"))
            .setSoReuseAddress(true)
            .setSoTimeout(configuration.getInt("proxy.list.socket.so.timeout"))
            .setTcpNoDelay(true).build();

    HttpClientBuilder builder = HttpClientBuilder.create();
    builder.disableAutomaticRetries();
    builder.disableContentCompression();
    builder.disableCookieManagement();
    builder.disableRedirectHandling();
    builder.setConnectionReuseStrategy(new NoConnectionReuseStrategy());
    builder.setDefaultSocketConfig(socketConfig);

One of my prototypes of doing shutdown:

      shutdownExecutor.schedule(() -> {
            httpClient.getConnectionManager().closeExpiredConnections();
            httpClient.getConnectionManager().closeIdleConnections(1, TimeUnit.SECONDS);
            httpClient.getConnectionManager().shutdown();

            httpClient.notifyAll();
            try {
                httpClient.close();
            } catch (IOException ex) {

            }

        }, configuration.getInt("proxy.test.forced.timeout.seconds"), TimeUnit.SECONDS);

        String content = HttpContentFetcher.getAndCloseClient(url, httpClient);

Can you post a minimal sample? How do you attempt to close connection and why do you state it has no effect? — Raffaele, Jan 10 '15 at 16:02
@Raffaele It stucks on reading response. Pasted details. Actually I;m pretty sure that I've achieved a timeout, but with length of about ~300 seconds, not 10 seconds as I want. — Piotr Müller, Jan 10 '15 at 18:10
It is kind of bizarre that you use socket linger in your code and expect the socket to get closed immediately at the same time. — ok2c, Jan 11 '15 at 15:39
@oleg Can you provide some details? I'm not aware what thus parameter exactly mean, but I just wanted it to have KNOWN value, to be sure that it is not 0=no limit. — Piotr Müller, Jan 12 '15 at 13:14
SO_LINGER parameter (including zero linger) and its relation to TIME_WAIT is explained reasonably well here: http://stackoverflow.com/questions/3757289/tcp-option-so-linger-zero-when-its-required — ok2c, Jan 12 '15 at 13:44

score 0 · Accepted Answer · answered Jan 10 '15 at 22:08

RequestConfig has helped. Now it looks like all the connections are discarded in specified limits.

       RequestConfig config= RequestConfig.custom()
                .setCircularRedirectsAllowed(false)
                .setConnectionRequestTimeout(4000)
                .setConnectTimeout(4000)
                .setMaxRedirects(0)
                .setRedirectsEnabled(false)
                .setSocketTimeout(4000)
                .setStaleConnectionCheckEnabled(true).build();
        request.setConfig(config);

How to force Apache HttpClient / HttpConnection to absolutely close tcp connection?

1 Answers1