Random occurrences of java.net.ConnectException

Question

I'm experiencing java.net.ConnectException in random ways.

My servlet runs in Tomcat 6.0 (JDK 1.6). The servlet periodically fetches data from 4-5 third-party web servers. The servlet uses a ScheduledExecutorService to fetch the data.

Run locally, all is fine and dandy. Run on my prod server, I see semi-random failures to fetch data from 1 of the third parties (Canadian weather data).

These are the URLs that are failing (plain RSS feeds):

Strange: each cycle, when I periodically fetch this data, the success/fail is all over the map: some succeed, some fail, but it never seems to be the same twice. So, I'm not completely blocked, just randomly blocked.

I slowed down my fetches, by introducing a 61s pause between each one. That had no effect.

The guts of the code that does the actual fetch:

  private static final int TIMEOUT = 60*1000; //msecs

  public String fetch(String aURL, String aEncoding /*UTF-8*/) {
    String result = "";
    long start = System.currentTimeMillis();
    Scanner scanner = null;
    URLConnection connection = null;
    try {
      URL url = new URL(aURL);
      connection =  url.openConnection(); //this doesn't talk to the network yet
      connection.setConnectTimeout(TIMEOUT); 
      connection.setReadTimeout(TIMEOUT);
      connection.connect(); //actually connects; this shouldn't be needed here
      scanner = new Scanner(connection.getInputStream(), aEncoding); 
      scanner.useDelimiter(END_OF_INPUT);
      result = scanner.next();
    }
    catch (IOException ex) {
      long end = System.currentTimeMillis();
      long time = end - start;
      fLogger.severe(
        "Problem connecting to " + aURL + " Encoding:" + aEncoding + 
        ". Exception: " + ex.getMessage() + " " + ex.toString() + " Cause:" +  ex.getCause() + 
        " Connection Timeout: " + connection.getConnectTimeout() + "msecs. Read timeout:" + 
        connection.getReadTimeout() + "msecs."
        + " Time taken to fail: " + time + " msecs."
      );
    }
    finally {
      if (scanner != null) scanner.close();
    }
    return result;
  }

Example log entry showing a failure:

SEVERE: Problem connecting to http://weather.gc.ca/rss/city/pe-5_e.xml Encoding:UTF-8. 
Exception: Connection timed out java.net.ConnectException: Connection timed out 
Cause:null 
Connection Timeout: 60000msecs. 
Read timeout:60000msecs. 
Time taken to fail: 15028 msecs.

Note that the time to fail is always 15s + a tiny amount. Also note that it fails to reach the configured 60s timeout for the connection.

The host-server admins (Environment Canada) state that they don't have any kind of a blacklist for the IP address of misbehaving clients.

Also important: the code had been running for several months without this happening.

http://stackoverflow.com/questions/5662283/java-net-connectexception-connection-timed-out-connect this might help — Karan Ashar, Jun 19 '15 at 21:06
Right, I saw that too. None of the possibilities there seems consistent with the randomness of the observed behaviour. — John, Jun 19 '15 at 21:09
Really hard to tell what's going on. Maybe a tcpdump on your server might tell you more. Sorry but I don't have any better suggestion :) — Karan Ashar, Jun 19 '15 at 21:35

score 0 · Answer 1 · answered Jun 22 '15 at 20:10

0

Someone suggested that instead I should use curl, a bash script, and cron. I implemented that, and it works fine.

I'm not able to solve this problem using Java.

answered Jun 22 '15 at 20:10

John

1,635
15
22

Random occurrences of java.net.ConnectException

1 Answers1