Spring Data Elasticsearch's @Field annotation not working

Question

I have a Spring Boot application with Spring Data Elasticsearch plugin in the pom.xml. I created a document class which i'd like to index:

@Document(indexName = "operations", type = "operation")
public class OperationDocument {

@Id
private Long id;

@Field(
    type = FieldType.String, 
    index = FieldIndex.analyzed, 
    searchAnalyzer = "standard", 
    indexAnalyzer = "standard",
    store = true
)
private String operationName;

@Field(
    type = FieldType.Date, 
    index = FieldIndex.not_analyzed, 
    store = true, 
    format = DateFormat.custom, pattern = "dd.MM.yyyy hh:mm"
)
private Date dateUp;

@Field(
    type = FieldType.String, 
    index = FieldIndex.not_analyzed, 
    store = false
) 
private String someTransientData;

@Field(type = FieldType.Nested)
private List<Sector> sectors;

//Getter and setters

I also created a repository for this class:

 public interface OperationDocumentRepository 
      extends ElasticsearchRepository<OperationDocument, Long> {
 }

I made a test that indexes three sample objects using the repository. It's pretty long so I'll post it only is needed. The fact is that the mapping created in the ES server ignores configuration set by @Field annotations:

"mappings": {
  "operation": {
    "properties": {
      "operationName": {
        "type": "string"
      },
      "dateUp": {
        "type": "long"
      },
      "someTransientData": {
        "type": "string"
      },
      "sectors": {
        "properties": {
          "id": {
            "type": "long"
          },
          "sectorName": {
            "type": "string"
          }
        }
      }
    }
  }
}

There is no info about analyzers, "someTransientData" is stored and indexed, and dateUp is typed as Long instead of Date.

A sample document requested directly from the server:

 {
   "_index": "operations",
   "_type": "operation",
   "_id": "AUyUk2cY3nXeOFxdOlQW",
   "_version": 1,
   "_score": 1,
   "_source": {
     "id": null,
     "operationName": "Second Operation Name",
     "dateUp": 1428421827091,
     "someTransientData": "Do not index or store",
     "sectors": [
       {
         "id": 2,
         "sectorName": "Health Care"
       },
       {
         "id": 3,
         "sectorName": "Construction"
       }
     ]
   }
 }

I also noted that when I run the application for the second time, at startup time I get this error, only printed when the index already exists:

ERROR 19452 --- [main] .d.e.r.s.AbstractElasticsearchRepository : failed to load elasticsearch nodes : org.elasticsearch.index.mapper.MergeMappingException: Merge failed with failures {[mapper [someTransientData] has different index values, mapper [someTransientData] has different tokenize values, mapper [someTransientData] has different index_analyzer, object mapping [sectors] can't be changed from non-nested to nested, mapper [operationName] has different store values, mapper [operationName] has different index_analyzer, mapper [dateUp] of different type, current_type [long], merged_type [date]]}

It's this a bug of Spring Data Elastic Search or I'm doing something wrong?

I tried the stable version provided by spring boot and last snapshot of spring-data-elasticsearch. I also tried the embedded Elasticsearch server provided by the plugin and an external one of the current version. I got always the same results.

Javier Alvarez · Accepted Answer · 2016-06-03T07:30:41.320

19

I could finally replicate and solve the problem. The fact is that I was using ElasticTemplate for indexing and searching docs instead of repositories, because my business logic got a more complicated (use of aggregations, etc.).

After that, I removed the unused OperationDocumentRespository. It seems that the repository is needed for the type mapping being posted to ES server on startup. I thought having the @Document class should be enough, but it isn't.

So we have two options here:

Keep the OperationDocumentRepository

Add this line to the app startup:

elasticsearchTemplate.putMapping(OperationDocument.class);

edited Jun 03 '16 at 07:30

answered Apr 24 '15 at 07:56

Javier Alvarez

1,409
2
15
33

I'm using SpringBoot - SpringData Elasticsearch v4.0.3 == ElasticSearch 7.6.2. Facing same issue @Field is not helping. I've created index using 'IndexOperations indexOps = elasticsearchTemplate.indexOps(Index.class); indexOps.create();' – SatyaRajC Oct 16 '20 at 10:28

Mohsin Husen · Answer 2 · 2015-04-10T15:28:02.653

I tried to replicate issue using spring data elasticsearch sample application but as per your configuration i am getting the desired results as mentioned above.

Entire code is committed to project here --> link

Look at the TestCase which generate index and apply mapping when spring context loaded--> link

Here is mapping generated by TestCase :

  {
  "operations" : {
    "aliases" : { },
    "mappings" : {
      "operation" : {
        "properties" : {
          "dateUp" : {
            "type" : "date",
            "store" : true,
            "format" : "dd.MM.yyyy hh:mm"
          },
          "operationName" : {
            "type" : "string",
            "store" : true,
            "analyzer" : "standard"
          },
          "sectors" : {
            "type" : "nested"
          },
          "someTransientData" : {
            "type" : "string",
            "index" : "not_analyzed"
          }
        }
      }
    },
    "settings" : {
      "index" : {
        "refresh_interval" : "1s",
        "number_of_shards" : "5",
        "store" : {
          "type" : "fs"
        },
        "creation_date" : "1428677234773",
        "number_of_replicas" : "1",
        "version" : {
          "created" : "1040499"
        },
        "uuid" : "-djzLu-IQ0CBs-M6R0-R6Q"
      }
    },
    "warmers" : { }
  }
}

Can you create similar example with spring boot using https://github.com/spring-projects/spring-boot/tree/master/spring-boot-samples/spring-boot-sample-data-elasticsearch

and commit on public share?

I tested your example and it worked. Then I created a new Spring Boot project and it worked too. Now I'm trying to figure what breaks my first application. I'll update the first post if I find it. Thank you! — Javier Alvarez, Apr 14 '15 at 11:20
I could finally replicate and solve the problem. I posted the reason and solution in another answer. — Javier Alvarez, Apr 24 '15 at 07:56

score 1 · Answer 3 · answered Sep 25 '20 at 06:29

I encountered this error as well using spring-data-elasticsearch and got a solution. Just put codes below, please check the comments. ES prohibited changing field types after indexes created. However you can change other attributes like fielddata.

Spring Data Elasticsearch 3.2.x
Elasticsearch 6.8.4
Spring Boot 2.2.x


package com.xxxx.xx.es.entity;

import lombok.Builder;
import lombok.Data;
import lombok.EqualsAndHashCode;
import org.springframework.data.annotation.Id;
import org.springframework.data.elasticsearch.annotations.Document;
import org.springframework.data.elasticsearch.annotations.Field;
import org.springframework.data.elasticsearch.annotations.FieldType;
import org.springframework.data.elasticsearch.annotations.Mapping;

@EqualsAndHashCode(callSuper = true)
@Data
@Builder
@Document(indexName = "gaming-laptop", indexStoreType = "fs")
@Mapping(mappingPath = "/gaming-laptop-mappings.json")   // use custom json to configure data structure in ES, default spring data won't work although using @Field(type = FieldType.Keyword)
public class GamingLaptop extends BaseEntity {

    @Id
    private String id;

    @Field
    private String cpu;

    @Field(type = FieldType.Keyword)  // this only works in ES 7.6.2+, Spring Data Elasticsearch 4.0.X+
    private String brandName;

    @Field
    private Integer cores;

    @Field
    private Integer threads;

    @Field
    private String coreFrequency;

    @Field
    private String ssd;

    @Field
    private String ram;

    @Field
    private String produceDate;

    @Field
    private Integer version;

    @Field
    private Double price;

    @Field(type = FieldType.Keyword)
    private String description;

}

resources/gaming-laptop-mappings.json

{
    "properties":{
        "_class":{
            "type":"text",
            "fields":{
                "keyword":{
                    "type":"keyword",
                    "ignore_above":256
                }
            }
        },
        "brandName":{
            "type":"keyword"
        },
        "coreFrequency":{
            "type":"text",
            "fields":{
                "keyword":{
                    "type":"keyword",
                    "ignore_above":256
                }
            }
        },
        "cores":{
            "type":"long"
        },
        "cpu":{
            "type":"text",
            "fields":{
                "keyword":{
                    "type":"keyword",
                    "ignore_above":256
                }
            }
        },
        "description":{
            "type":"keyword"
        },
        "gmtCreated":{
            "type":"text",
            "fields":{
                "keyword":{
                    "type":"keyword",
                    "ignore_above":256
                }
            }
        },
        "gmtModified":{
            "type":"text",
            "fields":{
                "keyword":{
                    "type":"keyword",
                    "ignore_above":256
                }
            }
        },
        "price":{
            "type":"float"
        },
        "produceDate":{
            "type":"text",
            "fields":{
                "keyword":{
                    "type":"keyword",
                    "ignore_above":256
                }
            }
        },
        "ram":{
            "type":"text",
            "fields":{
                "keyword":{
                    "type":"keyword",
                    "ignore_above":256
                }
            }
        },
        "ssd":{
            "type":"text",
            "fields":{
                "keyword":{
                    "type":"keyword",
                    "ignore_above":256
                }
            }
        },
        "threads":{
            "type":"long"
        },
        "version":{
            "type":"long"
        }
    }
}

Alternatively you can adapt Javier Alvarez's method by using rest high level client: elasticsearchTemplate.putMapping(OperationDocument.class);

Spring Data Elasticsearch's @Field annotation not working

3 Answers3

Linked