javax.net.ssl.SSLHandshakeException: "Remote host closed connection during handshake" opening an HTTPS page with HTMLUnit

Question

I'm trying to fetch the HTTPS page https://portal.mvp.bafin.de/database/AnteileInfo/ using HTML Unit 2.23 under Java 7, Ubuntu Linux.

The following code worked until some days ago, but now I receive the exception reported below.

The problem should be related to the type of security certificates used, but nothing changes executing the code with the vm option "-Dhttps.protocols=TLSv1.1,TLSv1.2" as I read here

The code:

import java.io.IOException;
import java.net.MalformedURLException;

import com.gargoylesoftware.htmlunit.FailingHttpStatusCodeException;
import com.gargoylesoftware.htmlunit.WebClient;
import com.gargoylesoftware.htmlunit.html.HtmlPage;

public class SSLTest {

    public static void main(String[] args) throws FailingHttpStatusCodeException, MalformedURLException, IOException {
        final String URL = "https://portal.mvp.bafin.de/database/AnteileInfo/";
        final WebClient wc = new WebClient();
        // the code should be ok without the following line,
        // but even with it, it doesn't work
        // wc.getOptions().setSSLClientProtocols(new String[]{"TLSv1.1","TLSv1.2"});
        final HtmlPage mainPage = wc.getPage(URL);
    }
}

The exception is the following: Remote host closed connection during handshake

Exception in thread "main" javax.net.ssl.SSLHandshakeException: Remote host closed connection during handshake
    at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:880)
    at sun.security.ssl.SSLSocketImpl.performInitialHandshake(SSLSocketImpl.java:1262)
    at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1289)
    at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1273)
    at org.apache.http.conn.ssl.SSLConnectionSocketFactory.createLayeredSocket(SSLConnectionSocketFactory.java:394)
    at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:353)
    at com.gargoylesoftware.htmlunit.httpclient.HtmlUnitSSLConnectionSocketFactory.connectSocket(HtmlUnitSSLConnectionSocketFactory.java:189)
    at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:141)
    at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:353)
    at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:380)
    at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236)
    at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:184)
    at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:88)
    at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110)
    at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:184)
    at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:71)
    at com.gargoylesoftware.htmlunit.HttpWebConnection.getResponse(HttpWebConnection.java:183)
    at com.gargoylesoftware.htmlunit.WebClient.loadWebResponseFromWebConnection(WebClient.java:1358)
    at com.gargoylesoftware.htmlunit.WebClient.loadWebResponse(WebClient.java:1275)
    at com.gargoylesoftware.htmlunit.WebClient.getPage(WebClient.java:382)
    at com.gargoylesoftware.htmlunit.WebClient.getPage(WebClient.java:304)
    at com.gargoylesoftware.htmlunit.WebClient.getPage(WebClient.java:451)
    at com.gargoylesoftware.htmlunit.WebClient.getPage(WebClient.java:436)
    at fetchers.bafin.SSLTest.main(SSLTest.java:16)
Caused by: java.io.EOFException: SSL peer shut down incorrectly
    at sun.security.ssl.InputRecord.read(InputRecord.java:352)
    at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:861)
    ... 23 more

if uncomment the following line

wc.getOptions().setSSLClientProtocols(new String[]{"TLSv1.1","TLSv1.2"});

I get the exception below, which gives a slightly different message: fatal alert: handshake_failure

Exception in thread "main" javax.net.ssl.SSLHandshakeException: Received fatal alert: handshake_failure
    at sun.security.ssl.Alerts.getSSLException(Alerts.java:192)
    at sun.security.ssl.Alerts.getSSLException(Alerts.java:154)
    at sun.security.ssl.SSLSocketImpl.recvAlert(SSLSocketImpl.java:1911)
    at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:1027)
[...]

Any suggestion?

Apparently, changing the version of Java to 1.8 could solve the problem : https://github.com/SeleniumHQ/htmlunit-driver/issues/8 — haihui, Jan 14 '17 at 06:37
thank you, I had already upgraded to java 8 in the meantime but it was the right answer — cdarwin, Jan 14 '17 at 11:48

cdarwin · Accepted Answer · 2017-01-14T11:05:35.950

The problem was solved upgrading to Java 8, as haihui said in a comment (but I had just upgraded before that comment).

I upgraded to Java 8 after reading Oracle's blog about HTTPS debugging and copying the JCE files suggested there in the $JAVA_HOME/lib/security (nothing changed).

I decided to upgrade to java 8 after I saw this site about SSL Site debugging couldn't connect using java 7 too.

During debugging I added this to the JRE arguments

-Djavax.net.debug=all

to see what's going on.

javax.net.ssl.SSLHandshakeException: "Remote host closed connection during handshake" opening an HTTPS page with HTMLUnit

1 Answers1