Allow insecure HTTPS connection for Java JDK 11 HttpClient

Question

Sometimes it is needed to allow insecure HTTPS connections, e.g. in some web-crawling applications which should work with any site. I used one such solution with old HttpsURLConnection API which was recently superseded by the new HttpClient API in JDK 11. What is the way to allow insecure HTTPS connections (self-signed or expired certificate) with this new API?

UPD: The code I tried (in Kotlin but maps directly to Java):

    val trustAllCerts = arrayOf<TrustManager>(object: X509TrustManager {
        override fun getAcceptedIssuers(): Array<X509Certificate>? = null
        override fun checkClientTrusted(certs: Array<X509Certificate>, authType: String) {}
        override fun checkServerTrusted(certs: Array<X509Certificate>, authType: String) {}
    })

    val sslContext = SSLContext.getInstance("SSL")
    sslContext.init(null, trustAllCerts, SecureRandom())

    val sslParams = SSLParameters()
    // This should prevent host validation
    sslParams.endpointIdentificationAlgorithm = ""

    httpClient = HttpClient.newBuilder()
        .sslContext(sslContext)
        .sslParameters(sslParams)
        .build()

But on sending I have exception (trying on localhost with self-signed certificate):

java.io.IOException: No name matching localhost found

Using IP address instead of localhost gives "No subject alternative names present" exception.

After some debugging of JDK I found that sslParams are really ignored in the place the exception is thrown and some locally created instance is used. Further debugging revealed that the only way to affect the hostname verification algorithm is setting jdk.internal.httpclient.disableHostnameVerification system property to true. And that's seems to be a solution. SSLParameters in the code above have no effect so this part can be discarded. Making it configurable only globally looks like serious design flaw in new HttpClient API.

Answer 1

As suggested already you need an SSLContext which ignores the bad certificates. The exact code which obtains the SSLContext in one of the links in the question should work by basically creating a null TrustManager which doesn't look at the certs:

private static TrustManager[] trustAllCerts = new TrustManager[]{
    new X509TrustManager() {
        public java.security.cert.X509Certificate[] getAcceptedIssuers() {
            return null;
        }
        public void checkClientTrusted(
            java.security.cert.X509Certificate[] certs, String authType) {
        }
        public void checkServerTrusted(
            java.security.cert.X509Certificate[] certs, String authType) {
        }
    }
};

public static  void main (String[] args) throws Exception {
    SSLContext sslContext = SSLContext.getInstance("TLS");
    sslContext.init(null, trustAllCerts, new SecureRandom());

    HttpClient client = HttpClient.newBuilder()
        .sslContext(sslContext)
        .build();

The problem with the above is obviously that server authentication is disabled completely for all sites. If there were only one bad certificate then you could import it into a keystore with:

keytool -importcert -keystore keystorename -storepass pass -alias cert -file certfile

and then initialize the SSLContext using an InputStream reading the keystore as follows:

char[] passphrase = ..
KeyStore ks = KeyStore.getInstance("PKCS12");
ks.load(i, passphrase); // i is an InputStream reading the keystore

KeyManagerFactory kmf = KeyManagerFactory.getInstance("PKIX");
kmf.init(ks, passphrase);

TrustManagerFactory tmf = TrustManagerFactory.getInstance("PKIX");
tmf.init(ks);

sslContext = SSLContext.getInstance("TLS");
sslContext.init(kmf.getKeyManagers(), tmf.getTrustManagers(), null);

Either of the above solutions will work for a self-signed certificate. A third option is in the case where the server provides a valid, non self-signed certificate but for a host which does not match any of the names in the certificate it provides, then a system property "jdk.internal.httpclient.disableHostnameVerification" can be set to "true" and this will force the certificate to be accepted in the same way that the HostnameVerifier API was used previously. Note, that in normal deployments it isn't expected that any of these mechanisms would be used, as it should be possible to automatically verify the certificate supplied by any correctly configured HTTPS server.

Answer 2

With Java 11, as well you can do a similar effort as mentioned in the selected answer in the link shared with the HttpClient built as:

HttpClient httpClient = HttpClient.newBuilder()
        .connectTimeout(Duration.ofMillis(<timeoutInSeconds> * 1000))
        .sslContext(sc) // SSL context 'sc' initialised as earlier
        .sslParameters(parameters) // ssl parameters if overriden
        .build();

with a sample request

HttpRequest requestBuilder = HttpRequest.newBuilder()
            .uri(URI.create("https://www.example.com/getSomething"))
            .GET()
            .build();

can be executed as:

httpClient.send(requestBuilder, HttpResponse.BodyHandlers.ofString()); // sends the request

---

Update from comments, to disable the hostname verification, currently one can use the system property:

-Djdk.internal.httpclient.disableHostnameVerification

which can be set programmatically as following :-

final Properties props = System.getProperties(); 
props.setProperty("jdk.internal.httpclient.disableHostnameVerification", Boolean.TRUE.toString());

Answer 3

Providing an X509TrustManager that does not do certificate verification (as in the answers above) is insufficient to disable hostname verification because the implementation of SSLContext "wraps" the provided X509TrustManager with an X509ExtendedTrustManager if it is not an X509ExtendedTrustManager. The wrapping "Extended" X509ExtendedTrustManager performs hostname verification.

A way to avoid hostname verification therefore is to provide an X509ExtendedTrustManager that does not do hostname verification. This may be done as follows:

import java.net.Socket;
import java.security.cert.X509Certificate;
import javax.net.ssl.SSLContext;
import javax.net.ssl.SSLEngine;
import javax.net.ssl.TrustManager;
import javax.net.ssl.X509ExtendedTrustManager;

SSLContext context = SSLContext.getInstance("TLS");
context.init(
    null,
    new TrustManager[]
    {
        new X509ExtendedTrustManager()
        {
            public X509Certificate[] getAcceptedIssuers()
            {
                return null;
            }

            public void checkClientTrusted(
                final X509Certificate[] a_certificates,
                final String a_auth_type)
            {
            }

            public void checkServerTrusted(
                final X509Certificate[] a_certificates,
                final String a_auth_type)
            {
            }
            public void checkClientTrusted(
                final X509Certificate[] a_certificates,
                final String a_auth_type,
                final Socket a_socket)
            {
            }
            public void checkServerTrusted(
                final X509Certificate[] a_certificates,
                final String a_auth_type,
                final Socket a_socket)
            {
            }
            public void checkClientTrusted(
                final X509Certificate[] a_certificates,
                final String a_auth_type,
                final SSLEngine a_engine)
            {
            }
            public void checkServerTrusted(
                final X509Certificate[] a_certificates,
                final String a_auth_type,
                final SSLEngine a_engine)
            {
            }
        }
    },
    null);

Supply this SSLContext to the HTTPClient.Builder in the usual way (as shown in earlier answers).

This method has the benefit of applying on a per HttpClient basis (i.e. not JVM-wide).