Getting jackson parsing error while serializing AggregatedPage in spring data elasticsearch

Question

I was trying to create a rest api point for aggregation in elasticsearch using spring data elasticsearch. I am able to get the data in service layer but when controller (i.e. @RestController) is trying to return it to postman i am getting this error.

Could not write JSON: For input string: "bHRMZzc5aHdodDF5a0hOck15Lzl1UT09"; nested exception is com.fasterxml.jackson.databind.JsonMappingException: For input string: "bHRMZzc5aHdodDF5a0hOck15Lzl1UT09" (through reference chain: org.springframework.data.elasticsearch.core.aggregation.impl.AggregatedPageImpl["aggregations"]->org.elasticsearch.search.aggregations.InternalAggregations["asMap"]->java.util.Collections$UnmodifiableMap["memberNumberToken"]->org.elasticsearch.search.aggregations.bucket.terms.StringTerms["buckets"]->java.util.ArrayList[0]->org.elasticsearch.search.aggregations.bucket.terms.StringTerms$Bucket["keyAsNumber"])

My Mapping file is this

{
  "transactions" : {
        "properties" : {

                "refernceId" : {
                  "type" :"text",
                  "index": true,
                  "store": true
                },
                "postingDate" : { 
                  "type" : "date",
                  "index": true,
                  "store": true,
                  "format" : "yyyy-MM-dd'T'HH:mm:ss'Z'"
                },
                "effectiveDate" : { 
                  "type" : "date",
                  "index": true,
                  "store": true,
                  "format" : "yyyy-MM-dd'T'HH:mm:ss'Z'"
                },
                "effectiveTime" : { 
                  "type" : "text",
                  "index": true,
                  "store": true
                },
                "effectiveEpochTime" : {
                  "type" : "long",
                  "index": true,
                  "store": true
                },
                "transactionAmount" : { 
                  "type" : "double",
                  "index": true,
                  "store": true
                },
                "transactionType" : { 
                  "type" : "keyword",
                  "index": true,
                  "store": true
                },
                "transactionDesc" : { 
                  "type" : "text",
                  "index": true,
                  "store": true
                },
                "transactionMemo" : {
                  "type" : "text",
                  "index": true,
                  "store": true
                },
                "transactionNumber" : { 
                  "type" : "text",
                  "index": true,
                  "store": true
                },
                "transactionTypeCode" : { 
                  "type" : "text",
                  "index": true,
                  "store": true
                },
                "transactionStatus" : {
                  "type" : "keyword",
                  "index": true,
                  "store": true
                },
                "principalAmount" : {
                  "type" : "double",
                  "index": true,
                  "store": true

                },
                "interest" : {
                  "type" : "text",
                  "index": true,
                  "store": true
                },
                "accountNumberToken" : { 
                  "type" : "keyword",
                  "index": true,
                  "store": true
                },
                "memberNumberToken" : {
                  "type" : "keyword",
                  "index": true,
                  "store": true
                },
                "accountType" : {
                  "type" : "keyword",
                  "index": true,
                  "store": true

                 },
                "userSub" : { 
                  "type" : "keyword",
                  "index": true,
                  "store": true
                },
                "tenant" : { 
                  "type" : "keyword",
                  "index": true,
                  "store": true
                }

            }
        }
    }

My pom file :

<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
    <modelVersion>4.0.0</modelVersion>
    <groupId>com.tyfone</groupId>
    <artifactId>mcb-search-feature</artifactId>
    <packaging>war</packaging>
    <version>0.0.1-SNAPSHOT</version>
    <name>mcb-search-feature Maven Webapp</name>
    <url>http://maven.apache.org</url>

    <properties>
        <jdk.version>1.8</jdk.version>
        <spring.version>5.0.1.RELEASE</spring.version>
        <jstl.version>1.2</jstl.version>
        <servletapi.version>4.0.0</servletapi.version>
        <spring.data.elastic.version>3.0.2.RELEASE</spring.data.elastic.version>
        <slf4j.version>1.7.25</slf4j.version>
        <junit.version>3.8.1</junit.version>

    </properties>

    <dependencies>




        <!-- Spring MVC framework -->
        <!-- https://mvnrepository.com/artifact/org.springframework/spring-webmvc -->
        <dependency>
            <groupId>org.springframework</groupId>
            <artifactId>spring-webmvc</artifactId>
            <version>${spring.version}</version>
        </dependency>

        <!-- https://mvnrepository.com/artifact/org.springframework/spring-web -->
        <!-- https://mvnrepository.com/artifact/org.springframework/spring-web -->
        <dependency>
            <groupId>org.springframework</groupId>
            <artifactId>spring-web</artifactId>
            <version>${spring.version}</version>
        </dependency>


        <!-- https://mvnrepository.com/artifact/org.springframework/spring-aop -->
        <dependency>
            <groupId>org.springframework</groupId>
            <artifactId>spring-aop</artifactId>
            <version>${spring.version}</version>
        </dependency>

        <!-- JSTL -->
        <dependency>
            <groupId>javax.servlet</groupId>
            <artifactId>jstl</artifactId>
            <version>${jstl.version}</version>
        </dependency>

        <!-- for compile only, your container should have this -->
        <!-- https://mvnrepository.com/artifact/javax.servlet/javax.servlet-api -->
        <dependency>
            <groupId>javax.servlet</groupId>
            <artifactId>javax.servlet-api</artifactId>
            <version>${servletapi.version}</version>
            <scope>provided</scope>
        </dependency>

        <!-- https://mvnrepository.com/artifact/org.springframework.data/spring-data-elasticsearch -->
        <dependency>
            <groupId>org.springframework.data</groupId>
            <artifactId>spring-data-elasticsearch</artifactId>
            <version>${spring.data.elastic.version}</version>
        </dependency>


        <!-- https://mvnrepository.com/artifact/org.slf4j/slf4j-simple -->
        <dependency>
            <groupId>org.slf4j</groupId>
            <artifactId>slf4j-simple</artifactId>
            <version>1.7.25</version>
            <scope>test</scope>
        </dependency>



        <!-- Test -->
        <dependency>
            <groupId>org.hamcrest</groupId>
            <artifactId>hamcrest-all</artifactId>
            <version>1.3</version>
            <scope>test</scope>
        </dependency>
        <!-- Testing pojo classes -->
        <dependency>
            <groupId>com.googlecode.openpojo</groupId>
            <artifactId>openpojo</artifactId>
            <version>0.6.0</version>
            <scope>test</scope>
        </dependency>
        <!-- <dependency> <groupId>junit</groupId> <artifactId>junit</artifactId> 
            <version>${junit.version}</version> <exclusions> <exclusion> <groupId>org.hamcrest</groupId> 
            <artifactId>hamcrest-core</artifactId> </exclusion> </exclusions> </dependency> -->

        <!-- https://mvnrepository.com/artifact/junit/junit -->
        <dependency>
            <groupId>junit</groupId>
            <artifactId>junit</artifactId>
            <version>4.12</version>
            <scope>test</scope>
        </dependency>


        <dependency>
            <groupId>org.springframework</groupId>
            <artifactId>spring-test</artifactId>
            <version>${spring.version}</version>
        </dependency>
        <dependency>
            <groupId>org.mockito</groupId>
            <artifactId>mockito-all</artifactId>
            <version>1.9.5</version>
            <scope>test</scope>
        </dependency>


    </dependencies>


    <build>
        <finalName>mcb-search-feature</finalName>
    </build>
</project>

Not really clear what you issue is. Can you describe a little bit more the context, what you're trying this achieve, etc... — Val, Dec 22 '17 at 08:13

hulk66 · Accepted Answer · 2018-03-20T15:26:24.740

If anyone is still interested. Had the exact same issue. It related to the fact that Jackson doesn't really know the correct types which are contained in AggregatedPage. So it simply runs through all the getters and stumbles over keyAsNumber where the key isn't a number, bHRMZzc5aHdodDF5a0hOck15Lzl1UT09 in your case. This ends up in an exception.

I ended up in configuring Jackson during the startup of the Spring App to omit keyAsNumber during the serialization.

So this did the job for me:

@Configuration
public class JacksonConfig   {

    @Bean
    public Jackson2ObjectMapperBuilderCustomizer changeKeyAsNumber() {
        return new Jackson2ObjectMapperBuilderCustomizer() {

            @Override
            public void customize(Jackson2ObjectMapperBuilder jacksonObjectMapperBuilder) {
                jacksonObjectMapperBuilder.mixIn(StringTerms.Bucket.class, MixIn.class);
            }
        };
    }

}

abstract class MixIn {
    @JsonIgnore
    abstract public Number getKeyAsNumber();
}

Is there any thing that need to do apart from above configuration? — Ramesh Papaganti, Aug 01 '19 at 13:57
Take a look at your exception line,.. The class StringTerms.Bucket.class may have to be replaced with ParsedStringTerms.ParsedBucket.class. This may be the case with newer versions of ElasticSearch library — Amaresh Kulkarni, Oct 05 '19 at 01:08

score 1 · Answer 2 · answered Jan 20 '21 at 20:53

I got it to work as Amaresh Kulkarni suggested to use the ParsedStringTerms.ParsedBucket.class instead of StringTerms.Bucket.class. Here is the complete JacksonConfiguraton

@Configuration
public class JacksonConfig   {

    @Bean
    public Jackson2ObjectMapperBuilderCustomizer changeKeyAsNumber() {
        return new Jackson2ObjectMapperBuilderCustomizer() {

            @Override
            public void customize(Jackson2ObjectMapperBuilder jacksonObjectMapperBuilder) {
                jacksonObjectMapperBuilder.mixIn(ParsedStringTerms.ParsedBucket.class, MixIn.class);
            }
        };
    }

}

abstract class MixIn {
    @JsonIgnore
    abstract public Number getKeyAsNumber();
}

This incantation worked for me. – Matt Friedman Sep 16 '21 at 12:23 — Matt Friedman, Sep 16 '21 at 12:23

score 0 · Answer 3 · answered Aug 25 '21 at 18:37

For anyone comming here after a search: I ran into the same problem and used WebFlux. Looks like it need an additional config. I merged the code from sung sung and the answer here: Configured ObjectMapper not used in spring-boot-webflux

My finished code looks like this:

@Configuration 
public class JacksonConfig{

@Bean
public Jackson2ObjectMapperBuilderCustomizer changeKeyAsNumber() {
    return jacksonObjectMapperBuilder -> jacksonObjectMapperBuilder.mixIn(ParsedStringTerms.ParsedBucket.class, MixIn.class);
}

@Bean
JavaTimeModule javaTimeModule(){
    return new JavaTimeModule();
}

@Bean
public Jackson2JsonEncoder jackson2JsonEncoder(ObjectMapper objectMapper){
    return new Jackson2JsonEncoder(objectMapper);
}

@Bean
public Jackson2JsonDecoder jackson2JsonDecoder(ObjectMapper objectMapper){
    return new Jackson2JsonDecoder(objectMapper);
}

@Bean
public WebFluxConfigurer webFluxConfigurer(Jackson2JsonEncoder encoder, Jackson2JsonDecoder decoder){
    return new WebFluxConfigurer() {
        @Override
        public void configureHttpMessageCodecs(ServerCodecConfigurer configurer) {
            configurer.defaultCodecs().jackson2JsonEncoder(encoder);
            configurer.defaultCodecs().jackson2JsonDecoder(decoder);
        }
    };
}

}

abstract class MixIn {
    @JsonIgnore
    abstract public Number getKeyAsNumber();
}

Getting jackson parsing error while serializing AggregatedPage in spring data elasticsearch

3 Answers3

Linked