Disabling SSL verification for Elastic search Restclient not working in Java

Question

I am trying to connect to elastic search hosted in a gcp box. To connect to this, there is a SSL check and I need a certificate.

However from these discussions, it is possible that we can turn off this verification.

How to disable SSL verification for Elasticsearch RestClient v6.7.0 in Java

https://discuss.elastic.co/t/host-name-does-not-match-the-certificate/186618

I had tried to remove the verification. But it throwing the following error

javax.net.ssl.SSLHandshakeException: General SSLEngine problem
    at org.elasticsearch.client.RestClient$SyncResponseListener.get(RestClient.java:947)
    at org.elasticsearch.client.RestClient.performRequest(RestClient.java:229)
    at org.elasticsearch.client.RestHighLevelClient.internalPerformRequest(RestHighLevelClient.java:1762)
    at org.elasticsearch.client.RestHighLevelClient.performRequest(RestHighLevelClient.java:1732)
    at org.elasticsearch.client.RestHighLevelClient.performRequestAndParseEntity(RestHighLevelClient.java:1694)
    at org.elasticsearch.client.RestHighLevelClient.search(RestHighLevelClient.java:1090)
    at org.dexter.lab.elasticUtils.ESUtils.getLastIndexedTimeStamp(ESUtils.java:44)
    at org.dexter.lab.druidUtils.DruidDelayChecker.main(DruidDelayChecker.java:357)
Caused by: javax.net.ssl.SSLHandshakeException: General SSLEngine problem
    at sun.security.ssl.Handshaker.checkThrown(Handshaker.java:1521)
    at sun.security.ssl.SSLEngineImpl.checkTaskThrown(SSLEngineImpl.java:528)
    at sun.security.ssl.SSLEngineImpl.writeAppRecord(SSLEngineImpl.java:1197)
    at sun.security.ssl.SSLEngineImpl.wrap(SSLEngineImpl.java:1165)
    at javax.net.ssl.SSLEngine.wrap(SSLEngine.java:469)
    at org.apache.http.nio.reactor.ssl.SSLIOSession.doWrap(SSLIOSession.java:265)
    at org.apache.http.nio.reactor.ssl.SSLIOSession.doHandshake(SSLIOSession.java:305)
    at org.apache.http.nio.reactor.ssl.SSLIOSession.isAppInputReady(SSLIOSession.java:509)
    at org.apache.http.impl.nio.reactor.AbstractIODispatch.inputReady(AbstractIODispatch.java:120)
    at org.apache.http.impl.nio.reactor.BaseIOReactor.readable(BaseIOReactor.java:162)
    at org.apache.http.impl.nio.reactor.AbstractIOReactor.processEvent(AbstractIOReactor.java:337)
    at org.apache.http.impl.nio.reactor.AbstractIOReactor.processEvents(AbstractIOReactor.java:315)
    at org.apache.http.impl.nio.reactor.AbstractIOReactor.execute(AbstractIOReactor.java:276)
    at org.apache.http.impl.nio.reactor.BaseIOReactor.execute(BaseIOReactor.java:104)
    at org.apache.http.impl.nio.reactor.AbstractMultiworkerIOReactor$Worker.run(AbstractMultiworkerIOReactor.java:588)
    at java.lang.Thread.run(Thread.java:748)
Caused by: javax.net.ssl.SSLHandshakeException: General SSLEngine problem
    at sun.security.ssl.Alerts.getSSLException(Alerts.java:192)
    at sun.security.ssl.SSLEngineImpl.fatal(SSLEngineImpl.java:1709)
    at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:318)
    at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:310)
    at sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1639)
    at sun.security.ssl.ClientHandshaker.processMessage(ClientHandshaker.java:223)
    at sun.security.ssl.Handshaker.processLoop(Handshaker.java:1037)
    at sun.security.ssl.Handshaker$1.run(Handshaker.java:970)
    at sun.security.ssl.Handshaker$1.run(Handshaker.java:967)
    at java.security.AccessController.doPrivileged(Native Method)
    at sun.security.ssl.Handshaker$DelegatedTask.run(Handshaker.java:1459)
    at org.apache.http.nio.reactor.ssl.SSLIOSession.doRunTask(SSLIOSession.java:283)
    at org.apache.http.nio.reactor.ssl.SSLIOSession.doHandshake(SSLIOSession.java:353)
    ... 9 more
Caused by: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
    at sun.security.validator.PKIXValidator.doBuild(PKIXValidator.java:397)
    at sun.security.validator.PKIXValidator.engineValidate(PKIXValidator.java:302)
    at sun.security.validator.Validator.validate(Validator.java:262)
    at sun.security.ssl.X509TrustManagerImpl.validate(X509TrustManagerImpl.java:324)
    at sun.security.ssl.X509TrustManagerImpl.checkTrusted(X509TrustManagerImpl.java:281)
    at sun.security.ssl.X509TrustManagerImpl.checkServerTrusted(X509TrustManagerImpl.java:136)
    at sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1626)
    ... 17 more
Caused by: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
    at sun.security.provider.certpath.SunCertPathBuilder.build(SunCertPathBuilder.java:141)
    at sun.security.provider.certpath.SunCertPathBuilder.engineBuild(SunCertPathBuilder.java:126)
    at java.security.cert.CertPathBuilder.build(CertPathBuilder.java:280)
    at sun.security.validator.PKIXValidator.doBuild(PKIXValidator.java:392)
    ... 23 more

This is what I did:

I did a override to verify function and returned true. So that it won't check. This is the code for that

restClientBuilder.setHttpClientConfigCallback(new RestClientBuilder.HttpClientConfigCallback() {
                @Override
                public HttpAsyncClientBuilder customizeHttpClient(HttpAsyncClientBuilder httpClientBuilder) {
                    return httpClientBuilder.setSSLHostnameVerifier(new HostnameVerifier() {
                        public boolean verify(String s, SSLSession sslSession) {
                            return true;
                        }
                    });
                }
            });

I tried with another way as well as mentioned in the elastic thread

restClientBuilder.setHttpClientConfigCallback(new RestClientBuilder.HttpClientConfigCallback() {
                @Override
                public HttpAsyncClientBuilder customizeHttpClient(HttpAsyncClientBuilder httpClientBuilder) {
                    return httpClientBuilder.setSSLHostnameVerifier(NoopHostnameVerifier.INSTANCE);
                }
            });

I had generated a self signed certificate and it worked. I had enabled debug logs to see why a certificate is needed.

These are the logs after adding certificate:

***
Found trusted certificate:
[
[
  Version: V3

This was followed by details about certificate.

The problem is I cannot generate self signed certificate for production environment and there I am getting exceptions for not using certificate even after overriding.

Is there any possible work around for this ? Any help is greatly appreciated.

Thanks in advance

Knowing [how many clusters have been compromised](https://www.linkedin.com/posts/valentincrettaz_what-is-elasticsearch-and-why-is-it-involved-activity-6729683899946958848-QITc) to date, I'm curious why you need to do this against a production box... that completely defeats the purpose of having SSL set up in the first place and opens your cluster to MITM attacks. Can you shed some light on why you need this? — Val, Nov 24 '20 at 04:46
Sure. I am making this connection from a production box (PB1). PB1 is isolated from the rest of the world. I am just querying the number of documents inserted per hour through a cron job from PB1. That's why I am looking at alternatives since the use case is small — Bharat, Nov 24 '20 at 04:59
It can't be compromised. That is behind a bastion host and it won't have any connectivity from outside — Bharat, Nov 24 '20 at 05:01
If there's one thing I've learned with security is that everything can be vulnerable at some point or another — Val, Nov 24 '20 at 05:02
Well. That is not impossible. But given the odds of having PB1 compromised and the use case I am looking for, I am looking for a way to override. If I am querying some sensitive information then definitely I need a certificate — Bharat, Nov 24 '20 at 05:07
Any chance that you mixed personal certificate with CA ? Because ' I need a certificate' typically means personal certificate, issued by CA trusted on both sides and used as authentication. — Alex Chernyshev, Nov 24 '20 at 05:11
@AlexChernyshev what I meant by "I need a certificate" is the public certificate that I can get from CA which is used on both sides for encrypting. — Bharat, Nov 24 '20 at 05:14
Ok, then issue could be in hostname, because server certificate is issued for fixed hostname or subdomain. Could you check that your client connects to elasticsearch instance by using same domain as in server's certificate? — Alex Chernyshev, Nov 24 '20 at 05:19
I currently do not have server's certificate. So I am not sure if I am connecting from the expected domain in server's certificate. But the verify method I am overriding is essentially to do this. It is to skip hostname verification and approve it — Bharat, Nov 24 '20 at 05:24
Sure, but as mentioned upper - removal of hostname verification is extremely bad idea on production system, so if you have issues with wrong domain - better to make correct mapping somewhere in /etc/hosts rather then disabling all verification :) — Alex Chernyshev, Nov 24 '20 at 05:31
Yeah you are right. But this is just a cron job so there won't be vulnerabilities. Anyways let me see how to use the certificate. Thanks @Val and AlexChernyshev — Bharat, Nov 24 '20 at 05:59

Hakan54 · Accepted Answer · 2022-01-05T23:50:53.550

In your example you only disabled the hostname verification. The server (ElasticSearch or something on top of ElasticSearch) is sending you the public key/certificate and your Restclient tries to validate that during the ssl handshake. What you need to do is tell to your RestClient that it is OK to receive a certificate from anyone but when it receives one it should not really validate it. So you need a custom trustmanager which has the task to validate the certificate but actually it doesn't validate at all. What you need is an UnsafeX509ExtendedTrustManager, see below for the code snippets and the usage with the RestClient:

Option 1

public final class UnsafeX509ExtendedTrustManager extends X509ExtendedTrustManager {

    private static final X509ExtendedTrustManager INSTANCE = new UnsafeX509ExtendedTrustManager();
    private static final X509Certificate[] EMPTY_CERTIFICATES = new X509Certificate[0];

    private UnsafeX509ExtendedTrustManager() {}

    public static X509ExtendedTrustManager getInstance() {
        return INSTANCE;
    }

    @Override
    public void checkClientTrusted(X509Certificate[] certificates, String authType) { 

    }

    @Override
    public void checkClientTrusted(X509Certificate[] certificates, String authType, Socket socket) {
    
    }

    @Override
    public void checkClientTrusted(X509Certificate[] certificates, String authType, SSLEngine sslEngine) {

    }

    @Override
    public void checkServerTrusted(X509Certificate[] certificates, String authType) {

    }

    @Override
    public void checkServerTrusted(X509Certificate[] certificates, String authType, Socket socket) {

    }

    @Override
    public void checkServerTrusted(X509Certificate[] certificates, String authType, SSLEngine sslEngine) {

    }

    @Override
    public X509Certificate[] getAcceptedIssuers() {
        return EMPTY_CERTIFICATES;
    }

}

The above trustmanager can be supplied to the RestHighLevelClient with the following snippet:

SSLContext sslContext = SSLContext.getInstance("TLS");
sslContext.init(null, new TrustManager[]{ UnsafeX509ExtendedTrustManager.INSTANCE }, null);

RestClient restClient = RestClient
        .builder(new HttpHost("localhost", 9200, "https"))
        .setHttpClientConfigCallback(httpClientBuilder -> 
                httpClientBuilder.setSSLContext(sslContext)
                                 .setSSLHostnameVerifier((host, session) -> true))
        .build();

By the way, I don't recommend you or anyone else to use UnsafeX509ExtendedTrustManager. It is unsafe and should not be used at all in production.

Option 2

If you don't want to add the custom code to your code base but just only want to easily disable the ssl verification, you might want to give the following snippet a try. It is a library to easily generate the SSLContext or other ssl materials and it has the option to disable the ssl verification.

<dependency>
    <groupId>io.github.hakky54</groupId>
    <artifactId>sslcontext-kickstart</artifactId>
    <version>7.1.0</version>
</dependency>

Usage

SSLFactory sslFactory = SSLFactory.builder()
          .withUnsafeTrustMaterial()
          .withUnsafeHostnameVerifier()
          .build();

RestClient restClient = RestClient
        .builder(new HttpHost("localhost", 9200, "https"))
        .setHttpClientConfigCallback(httpClientBuilder -> 
                httpClientBuilder.setSSLContext(sslFactory.getSslContext())
                                 .setSSLHostnameVerifier(sslFactory.getHostnameVerifier())
        .build();

Awesome, did you also tried without the custom hostname verifier? Does that work? — Hakan54, Nov 24 '20 at 14:49
Yes. I tried with removing custom hostname verifier. That also worked !! — Bharat, Nov 24 '20 at 16:16
It hasn't worked without custom hostname in a different environment. I had to use both methods to make it work — Bharat, Nov 25 '20 at 07:09

Anusree · Answer 2 · 2021-05-04T15:26:15.347

Hi there is very much easy way to do this. with less code.

hope this will help you, I had the same problem and this is how I resolved.

    @Bean
        public RestHighLevelClient createSimpleElasticClient() throws Exception {
            try {
                SSLContextBuilder sslBuilder = SSLContexts.custom()
                        .loadTrustMaterial(null, (x509Certificates, s) -> true);
                        final SSLContext sslContext = sslBuilder.build();
                RestHighLevelClient client = new RestHighLevelClient(RestClient
                        .builder(new HttpHost(hostNameOrLoadbalancerURL, 443, "https")) 
//port number is given as 443 since its https schema
                        .setHttpClientConfigCallback(new HttpClientConfigCallback() {
                            @Override
                            public HttpAsyncClientBuilder customizeHttpClient(HttpAsyncClientBuilder httpClientBuilder) {
                                return httpClientBuilder
                                         .setSSLContext(sslContext)
                                         .setSSLHostnameVerifier(NoopHostnameVerifier.INSTANCE);
                            }
                        })
                        .setRequestConfigCallback(new RestClientBuilder.RequestConfigCallback() {
                            @Override
                            public RequestConfig.Builder customizeRequestConfig(
                                    RequestConfig.Builder requestConfigBuilder) {
                                return requestConfigBuilder.setConnectTimeout(5000)
                                        .setSocketTimeout(120000);
                            }
                        }));
                System.out.println("elasticsearch client created");
                return client;
            } catch (Exception e) {
                System.out.println(e);
                throw new Exception("Could not create an elasticsearch client!!");
            }
        }

Disabling SSL verification for Elastic search Restclient not working in Java

2 Answers2

Linked