Elasticsearch 8.3 configuration in Spring Boot

Question

I have been trying to implement Elastic Search 8.3 with spring boot.

i am getting the below exception:

Cannot convert value of type 'co.elastic.clients.elasticsearch.ElasticsearchClient' to required type 'org.springframework.data.elasticsearch.core.ElasticsearchOperations'

Below is my configuration for elastic search:

import org.apache.http.HttpHost;
import org.apache.http.auth.AuthScope;
import org.apache.http.auth.UsernamePasswordCredentials;
import org.apache.http.client.CredentialsProvider;
import org.apache.http.impl.client.BasicCredentialsProvider;
import org.apache.http.impl.nio.client.HttpAsyncClientBuilder;
import org.elasticsearch.client.RestClient;
import org.elasticsearch.client.RestClientBuilder.HttpClientConfigCallback;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.data.elasticsearch.repository.config.EnableElasticsearchRepositories;

import co.elastic.clients.elasticsearch.ElasticsearchClient;
import co.elastic.clients.json.jackson.JacksonJsonpMapper;
import co.elastic.clients.transport.ElasticsearchTransport;
import co.elastic.clients.transport.rest_client.RestClientTransport;
@Configuration
@EnableElasticsearchRepositories
public class ElasticConfig {

    @Bean
    public RestClient client() {
        final CredentialsProvider credentialsProvider = new BasicCredentialsProvider();
        credentialsProvider.setCredentials(AuthScope.ANY, new UsernamePasswordCredentials("elastic", "test321"));

        HttpHost host = new HttpHost("localhost", 9200);

        RestClient restClient = RestClient.builder(host)
                .setHttpClientConfigCallback(new HttpClientConfigCallback() {

                    @Override
                    public HttpAsyncClientBuilder customizeHttpClient(HttpAsyncClientBuilder httpClientBuilder) {
                        // TODO Auto-generated method stub
                        httpClientBuilder.disableAuthCaching();
                        return httpClientBuilder.setDefaultCredentialsProvider(credentialsProvider);
                    }
                }).build();
        return restClient;
    }

    @Bean
    public ElasticsearchClient elasticsearchTemplate() {
        ElasticsearchTransport transport = new RestClientTransport(client(), new JacksonJsonpMapper());
        return new ElasticsearchClient(transport);
    } 
    
}

Below is POM:

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>
    <parent>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter-parent</artifactId>
        <version>2.7.2</version>
        <relativePath /> <!-- lookup parent from repository -->
    </parent>
    <groupId>aisha</groupId>
    <artifactId>courseapp</artifactId>
    <version>0.0.1-SNAPSHOT</version>
    <packaging>war</packaging>
    <name>courseapp</name>
    <description>Demo project for Spring Boot</description>
    <properties>
        <java.version>17</java.version>
        <guava.version>20.0</guava.version>
        <geoip2.version>2.15.0</geoip2.version>
        <uap-java.version>1.4.0</uap-java.version>

    </properties>
    <dependencies>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-web</artifactId>
        </dependency>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-websocket</artifactId>
        </dependency>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-data-jdbc</artifactId>
        </dependency>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-devtools</artifactId>
            <scope>runtime</scope>
            <optional>true</optional>
        </dependency>
        <dependency>
            <groupId>org.postgresql</groupId>
            <artifactId>postgresql</artifactId>
            <scope>runtime</scope>
        </dependency>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-tomcat</artifactId>
            <scope>provided</scope>
        </dependency>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-test</artifactId>
            <scope>test</scope>
        </dependency>
        <dependency>
            <groupId>org.junit.platform</groupId>
            <artifactId>junit-platform-suite</artifactId>
            <scope>test</scope>
        </dependency>
        <dependency>
            <groupId>org.apache.tomcat.embed</groupId>
            <artifactId>tomcat-embed-jasper</artifactId>
        </dependency>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-security</artifactId>
        </dependency>
        <dependency>
            <groupId>javax.servlet</groupId>
            <artifactId>jstl</artifactId>
        </dependency>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-validation</artifactId>
        </dependency>
        <dependency>
            <groupId>com.google.guava</groupId>
            <artifactId>guava</artifactId>
            <version>${guava.version}</version>
        </dependency>
        <dependency>
            <groupId>com.maxmind.geoip2</groupId>
            <artifactId>geoip2</artifactId>
            <version>${geoip2.version}</version>
        </dependency>
        <!-- https://mvnrepository.com/artifact/com.google.code.gson/gson -->
        <dependency>
            <groupId>com.google.code.gson</groupId>
            <artifactId>gson</artifactId>
        </dependency>
        
        
         <dependency>
            <groupId>com.github.ua-parser</groupId>
            <artifactId>uap-java</artifactId>
            <version>${uap-java.version}</version>
        </dependency>
        <dependency>
    <groupId>org.springframework.data</groupId>
    <artifactId>spring-data-elasticsearch</artifactId>
        </dependency>
        
          <dependency>
      <groupId>co.elastic.clients</groupId>
      <artifactId>elasticsearch-java</artifactId>
      <version>8.3.3</version>
    </dependency>

    <dependency>
      <groupId>com.fasterxml.jackson.core</groupId>
      <artifactId>jackson-databind</artifactId>
    </dependency>
     <dependency>
      <groupId>jakarta.json</groupId>
      <artifactId>jakarta.json-api</artifactId>
      <version>2.0.1</version>
    </dependency>
   

<!-- https://mvnrepository.com/artifact/com.github.javafaker/javafaker -->
<dependency>
    <groupId>com.github.javafaker</groupId>
    <artifactId>javafaker</artifactId>
    <version>1.0.2</version>
</dependency>

    </dependencies>

    <build>
        <plugins>
            <plugin>
                <groupId>org.springframework.boot</groupId>
                <artifactId>spring-boot-maven-plugin</artifactId>
            </plugin>
        </plugins>
    </build>

</project>

Below is test class which call simple query for index product :

@SpringBootTest

public class ElkSpringTest {

     @Autowired
        private ElasticsearchRestTemplate elasticConfig;
    
    
    
    @Test
    void contextLoads() {
        SearchResponse<ELKProduct> search;

        try {
            search = elasticConfig.search(s -> s.index("product").size(10), ELKProduct.class);
            System.out.println(search.hits().total().value());

            for (Hit<ELKProduct> hit : search.hits().hits()) {
                System.out.println(hit.source().getId());
            }

            System.out.println("******ENding *****");
        } catch (ElasticsearchException | IOException e) {
            // TODO Auto-generated catch block
            e.printStackTrace();
        }

        
        
        
    }
}

Since elastic is upgraded to latest version 8.3. I am not able to set up elastic search in spring boot. I usually work with ElasticSearch repository to store the data in my indexs and query it using ElasticsearchOperations.

why you are using `spring-data-elasticsearch` and `elasticsearch-java` both in your pom.xml file ? You can use any one to connect to the Elasticsearch. Also, `spring-data-elasticsearch` already have dependancy for Elasticsearch rest client so if you want to use Spring then only spring data dependancy is sufficiant. You are getting error because of mismatch of dependancy while using class or object. — Sagar Patel, Aug 16 '22 at 09:25

score 1 · Answer 1 · answered Nov 17 '22 at 07:44

Current spring-data-elasticsearch(version 4.4) not much compatible with es version 8.3 (>7.17.6).

ES RestHighLevelClient can setApiCompatibilityMode(true) to enables compatibility mode that allows HLRC 7.17 to work with Elasticsearch 8.x. But the compatibility is not very good.

score 0 · Answer 2 · answered Jan 06 '23 at 12:10

0

spring-data-elasticsearch(version 5.0) is now compatible with ES 8.5.0

It looks like the RestHighLevelClient is deprecated as of this version.

Check this documentation for implementing the client configuration.

answered Jan 06 '23 at 12:10

Iulian Masar

1

While this link may answer the question, it is better to include the essential parts of the answer here and provide the link for reference. Link-only answers can become invalid if the linked page changes. - [From Review](/review/late-answers/33573977) – Bö macht Blau Jan 08 '23 at 09:30

score 0 · Answer 3 · answered Jul 28 '23 at 13:55

On the date of this answer, I am currently able to config Java API client (provided officially by Elasticsearch) to works with ElasticSearch 8.3.x

Version of dependencies used (if it helps):

org.springframework.boot:spring-boot-starter:2.7.13
org.springframework.data:spring-data-elasticsearch:4.4.13
co.elastic.clients:elasticsearch-java:7.17.9 (depended by the above one).

Your ElasticConfig (with the annotation @Configuration) class just needs to extend ElasticsearchConfiguration abstract class.

Something similar to this

@Configuration
public class ElasticConfig extends ElasticsearchConfiguration {
    @Override
    public ClientConfiguration clientConfiguration() {
        try {

            // TODO: Should be on the dev profile to run this on self-signed TLS server.
            final CredentialsProvider credentialsProvider = new BasicCredentialsProvider();
            credentialsProvider.setCredentials(AuthScope.ANY, new UsernamePasswordCredentials(this.esUsername, this.esPassword));

            SSLContextBuilder sslBuilder = SSLContexts.custom()
                    .loadTrustMaterial(null,
                            (x509Certificates, s) -> true
                    );
            final SSLContext sslContext = sslBuilder.build();

            return ClientConfiguration.builder()
                    .connectedTo(esUris)
                    .usingSsl(sslContext, NoopHostnameVerifier.INSTANCE)
                    .withConnectTimeout(10000)
                    .withBasicAuth(esUsername, esPassword)
                    .build();
        } catch (Exception e) {
            // throwing properly
            return null;
        }

    }
}

The abstract class has (pre)-declared enough Bean for our job. We only need to declare some Operations or Repository to do the job.

Adapt your esUris, esUsername, esPassword with yours or eliminate them if you're using an unsecure local ES server.

Elasticsearch 8.3 configuration in Spring Boot

3 Answers3