Java HTTPUrlConnection timeout does not work

Question

I've written a programm that opens a httpurlconnection to a website through random proxies. My httpurlconnection is called conn. Now I know, that some of those proxies might be too slow, so i've set the timeout of the connection to 40000 milliseconds with conn.setConnectTimeout(40000) and conn.setReadTimeout(40000).

After doing so, i got this code:

long diff = 0;
    long starttime = 0;
    long endtime = 0;

    try
    {
        starttime = System.currentTimeMillis();
        conn.connect();

        endtime = System.currentTimeMillis();

        diff = endtime - starttime;

        if (endtime <= starttime + conn.getConnectTimeout())
        {
            //Trying to read sourecode
            InputStreamReader isrConn = new InputStreamReader(conn.getInputStream());
            BufferedReader brConn = new BufferedReader(isrConn);

            line = brConn.readLine();

            while (line != null)
            {

                response += line + "\t";
                try
                {
                    line = brConn.readLine();
                } catch (IOException e)
                {
                    printError("Reading sourcecode failed.");
                }

            }
        }
        else
        {
            response = "blabla.";
        }




    // If conn.connect failed   
    } catch (IOException e)
    {
        endtime = System.currentTimeMillis();
        diff = endtime - starttime;

        response = "Message: "+e.getMessage() +" MyTimeout:"+ conn.getConnectTimeout() +" Actual time passed:  "+ diff;
               e.printStackTrace();


    }

There are reasons why the connection could fail, so in many cases i get to the last catch-block and get the following output:

Message: Connection timed out: connect MyTimeout:40000 Actual time passed: 21012

Message: Connection timed out: connect MyTimeout:40000 Actual time passed: 21016

Message: Connection timed out: connect MyTimeout:40000 Actual time passed: 21010

Message: Connection timed out: connect MyTimeout:40000 Actual time passed: 21009

So my question would be: I have set the timeout to 40000 milliseconds, but i get a "Connection timed out"-response after about 21000 milliseconds, does any of you know why that is?

EDIT: im using windows 7 and i now added the e.printStackTrace() to the catch-block, like told in the comments. thanks so far. the output now is (example):

java.net.ConnectException: Connection timed out: connect
    at java.net.DualStackPlainSocketImpl.waitForConnect(Native Method)
    at java.net.DualStackPlainSocketImpl.socketConnect(Unknown Source)
    at java.net.AbstractPlainSocketImpl.doConnect(Unknown Source)
    at java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source)
    at java.net.AbstractPlainSocketImpl.connect(Unknown Source)
    at java.net.PlainSocketImpl.connect(Unknown Source)
    at java.net.Socket.connect(Unknown Source)
    at sun.net.NetworkClient.doConnect(Unknown Source)
    at sun.net.www.http.HttpClient.openServer(Unknown Source)
    at sun.net.www.http.HttpClient$1.run(Unknown Source)
    at sun.net.www.http.HttpClient$1.run(Unknown Source)
    at java.security.AccessController.doPrivileged(Native Method)
    at sun.net.www.http.HttpClient.privilegedOpenServer(Unknown Source)
    at sun.net.www.http.HttpClient.openServer(Unknown Source)
    at sun.net.www.http.HttpClient.<init>(Unknown Source)
    at sun.net.www.http.HttpClient.New(Unknown Source)
    at sun.net.www.http.HttpClient.New(Unknown Source)
    at sun.net.www.protocol.http.HttpURLConnection.getNewHttpClient(Unknown Source)
    at sun.net.www.protocol.http.HttpURLConnection.plainConnect(Unknown Source)
    at sun.net.www.protocol.http.HttpURLConnection.connect(Unknown Source)
    at TestThread.getSourcePage(TestThread.java:361)
    at TestThread.aChecker(TestThread.java:216)
    at TestThread.getNextProxy(TestThread.java:169)
    at TestThread.getNextC(TestThread.java:157)
    at TestThread.aChecker(TestThread.java:273)
    at TestThread.getNextProxy(TestThread.java:169)
    at TestThread.aChecker(TestThread.java:295)
    at TestThread.getNextProxy(TestThread.java:169)
    at TestThread.getNextC(TestThread.java:157)
    at TestThread.run(TestThread.java:103)
    at java.lang.Thread.run(Unknown Source)

Message: Connection timed out: connect MyTimeout:40000 Actual time passed:  21015

Please output and post the actual stack trace instead of your own message. — Brian Roach, Dec 24 '12 at 22:54
As mentioned by Brian Roach, please post the output from `e.printStackTrace()` in the catch block. — Chris Snow, Dec 24 '12 at 23:17
added what you said. i hope you can help me further, thanks in advance. — user1927372, Dec 24 '12 at 23:41
The javadoc for URLConnection's `.setConnectTimeout()` explicitly says that "Some non-standard implmentation [sic] of this method may ignore the specified timeout" and recommends to check the actual value by calling `.getConnectTimeout()` <-- what does this method say after your attempt to set it? — fge, Dec 25 '12 at 00:10
You could try values < 21000 too, to see whether 21000 is determined by an extra constraint of the system. — Joop Eggen, Dec 25 '12 at 00:11
21 seconds is because of windows: http://superuser.com/questions/339959/how-to-set-tcp-ip-abort-interval-or-timeout-in-windows-xp (those are the same values used in Windows 7 as well). Note that usually when you're setting a connection timeout it's because you want to timeout in a shorter amount of time than the system default. — Brian Roach, Dec 25 '12 at 03:36
At fge: im setting the timeout with conn.setConnTimeout(40000) and when I'm calling conn.getConnTimeout(), like I do in the catch-block, the output is 40000. Thats why I thought the implementation worked. @Brian Roach: thanks for the explaination, but does that mean I can not set a timeout higher than 21 seconds in my java program if I really want to? — user1927372, Dec 25 '12 at 09:21
Correct - the underlying TCP implementation in the OS is going to time out at 21 seconds *max*. — Brian Roach, Dec 25 '12 at 14:28
ok, thanks for the clarification. i hoped that there was a way to increase the timeout. alright, i gotta accept that then ;) thanks again. — user1927372, Dec 25 '12 at 16:26
Depending on the situation (i.e. if you control the environment the app is running in or not) you will be able to reconfigure the time-out in the OS. — Tim B, Dec 20 '13 at 10:00

score 11 · Answer 1 · answered Dec 23 '13 at 21:20

Look at the exception you got:
The biggest clue: You are getting java.net.ConnectException As per javadoc, java.net.ConnectException signifies that connection was refused remotely for reasons such as no process is listening on the port.

public class ConnectException
extends SocketException

Signals that an error occurred while attempting to connect a socket to a remote address and port. 
Typically, the connection was refused remotely (e.g., no process is listening on the remote 
address/port)

What you configured in HttpUrlConnection:
The timeout for connection (given that the remote port accepts connection). If the connection timeout expires, you get a java.net.SocketTimeoutException and not a java.net.ConnectException.

So, what is causing java.net.ConnectException?
I tried the following test cases:

   +------------+------------+----------------+------------------+---------------------------------+ 
   | Valid Host | Valid Port | Valid Proxy IP | Valid Proxy Port | Exception                       | 
   +------------+------------+----------------+------------------+---------------------------------+ 
#1 | yes        | yes        | -NA-           | -NA-             | -- none --                      |
#2 | yes        | no         | -NA-           | -NA-             | java.net.ConnectException       |
   +------------+------------+----------------+------------------+---------------------------------+ 
#3 | yes        | yes        | yes            | yes              | -- none --                      |
#4 | yes        | no         | yes            | yes              | java.net.SocketTimeoutException |
#5 | yes        | yes        | yes            | no               | java.net.ConnectException       |
   +------------+------------+----------------+------------------+---------------------------------+

Case #1, #3 are the happy paths in which all configs are correct
In case #4, we get a java.net.SocketTimeoutException because java process is able to establish connection (to the proxy port), but does not get any data to read as target host's port number is invalid
In case #2, #5 we get java.net.ConnectException because the port at which java process attempts to write/read is invalid
The connection timeout value does not hold good for cases where no process is listening on the port at which java process is attempting to connect. That's why you get ConnectException before timeout expires

Message: Connection refused: connect MyTimeout:10000 Actual time passed: 6549 java.net.ConnectException: Connection refused: connect at java.net.DualStackPlainSocketImpl.waitForConnect(Native Method) at java.net.DualStackPlainSocketImpl.socketConnect(DualStackPlainSocketImpl.java:85) .... ....

Conclusion:

Some of the proxies you were trying to connect to, must be down. Hence java process threw java.net.ConnectException
It's better to catch java.net.ConnectException and mark the proxy as invalid/down

score 6 · Answer 2 · answered Jun 04 '13 at 21:14

In my experience, HttpUrlConnection will not timeout during name resolution. If your device has cached the target address, then it will timeout correctly.

For testing purposes use your ip address in your code.

This usually happens to me on mobile devices.

Java HTTPUrlConnection timeout does not work

2 Answers2

Linked