Apache http client -> local address bind -> BindException after some time

Question

OS: Linux ( Debian )
Java: Java(TM) SE Runtime Environment (build 1.7.0_05-b06)
Java: Java HotSpot(TM) 64-Bit Server VM (build 23.1-b03, mixed mode)
Apache http client: v4.2.3 ( latest )

I want to create one PoolingConnectionManager with several clients, each with its unique local address (ConnRoutePNames.LOCAL_ADDRESS).
Those clients will be used by several workers ( each worker selects random client and executes request ).

The problem is: When i set local address for clients, after some time (e.g. 1 min) i always get java.net.BindException with message "Address already in use".

Question: Is it bug?

java.net.BindException: Address already in use
at java.net.PlainSocketImpl.socketBind(Native Method) ~[na:1.7.0_05]
at java.net.AbstractPlainSocketImpl.bind(AbstractPlainSocketImpl.java:376) ~[na:1.7.0_05]
at java.net.Socket.bind(Socket.java:627) ~[na:1.7.0_05]
at org.apache.http.conn.scheme.PlainSocketFactory.connectSocket(PlainSocketFactory.java:120) ~[httpclient-4.2.3.jar:4.2.3]
at org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:180) ~[httpclient-4.2.3.jar:4.2.3]
at org.apache.http.impl.conn.ManagedClientConnectionImpl.open(ManagedClientConnectionImpl.java:294) ~[httpclient-4.2.3.jar:4.2.3]
at org.apache.http.impl.client.DefaultRequestDirector.tryConnect(DefaultRequestDirector.java:645) ~[httpclient-4.2.3.jar:4.2.3]
at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:480) ~[httpclient-4.2.3.jar:4.2.3]
at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:906) ~[httpclient-4.2.3.jar:4.2.3]
at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:805) ~[httpclient-4.2.3.jar:4.2.3]
at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:784) ~[httpclient-4.2.3.jar:4.2.3]
at controllers.Test$Worker.run(Test.java:67) ~[test_2.9.1-1.0-SNAPSHOT.jar:1.0-SNAPSHOT]
at java.lang.Thread.run(Thread.java:722) [na:1.7.0_05]

Code ( simplified ):

public static void main(String[] args) throws UnknownHostException
{
    ClientConnectionManager connManager = buildConnectionManager(30);

    List<HttpClient> clients = new ArrayList<>();
    for(int i=0; i<8; ++i)
    {
        HttpClient client = buildClient(connManager, InetAddress.getByName("111.111.111." + (50 + i));
        clients.add(client);
    }

    for(int i=0; i<30; ++i)
    {
        new Thread(new Worker(clients)).start();
    }
}

public static ClientConnectionManager buildConnectionManager(Integer parallelism)
{
    PoolingClientConnectionManager connectionManager = new PoolingClientConnectionManager();
    connectionManager.setMaxTotal(parallelism);
    connectionManager.setDefaultMaxPerRoute(parallelism);

    return connectionManager;
}

public static HttpClient buildClient(ClientConnectionManager connectionManager, InetAddress localAddress)
{
    DefaultHttpClient httpClient = new DefaultHttpClient(connectionManager);
    HttpParams params = httpClient.getParams();
    params.setParameter(ConnRoutePNames.LOCAL_ADDRESS, localAddress);

    return httpClient;
}

private static class Worker implements Runnable
{
    private List<HttpClient> clients = null;

    public Worker(List<HttpClient> clients)
    {
        this.clients = clients;
    }

    public void run()
    {

        do
        {
            HttpGet httpGet = new HttpGet("http://google.com/robots.txt");
            HttpClient client = this.clients.get(new Random().nextInt(this.clients.size()));

            try
            {
                HttpResponse httpResponse = client.execute(httpGet);
                EntityUtils.consume(httpResponse.getEntity());

                logger.debug("Success request");
            }
            catch(IOException e)
            {
                httpGet.abort();
                logger.info("IO error", e);
            }
        }
        while(true);
    }
}

Try a `httpResponse.releaseConnection()` after consuming the response. Ideally in a finally block as shown [here](http://hc.apache.org/httpcomponents-client-ga/quickstart.html) — Pyranja, Jan 20 '13 at 01:20
@Pyranja, no, it does not work. Anyway, it would not be good solve of my problem :( — Oleg Golovanov, Jan 20 '13 at 10:03

score 3 · Answer 1 · answered Jan 21 '13 at 10:04

3

This issue may have to do with your TCP stack. I don't think this is a bug in HttpClient. You may find the solution here: http://planet.jboss.org/post/concurrent_high_throughput_performance_testing_with_jmeter

answered Jan 21 '13 at 10:04

Francois-Xavier Bonnet

31
1

score 1 · Answer 2 · answered Jan 20 '13 at 19:40

1

The only theory I can think of is that the TCP/IP stack runs out of ports and starts assigning the same port numbers to new connections while old connections have not been fully cleaned up. I see no evidence of this problem being a bug in HttpClient.

answered Jan 20 '13 at 19:40

ok2c

26,450
5
63
71

score 0 · Accepted Answer · answered Jan 21 '13 at 15:49

Thanks for your answers!
Yes, the problem was with tcp/ip stack, but i was unable to get why :)

So, the reason explained below.

I created connection manager ( e.g. size=10 ), several clients ( each with its unique local address ), and several workers.
Each worker takes random client and executes the request.
So, at one time its possible that one client can be executed by up to 10 workers in parallel.
At another time another client can be executed by up to 10 workers in parallel.
Thats why, all 10 connections in manager can not be keep-alived, they are closing/creating all the time.

Sorry for bothering you!

Apache http client -> local address bind -> BindException after some time

3 Answers3

Linked