Getting count and list of ids using ElasticsearchTemplate in spring-data-elasticsearch

Question

I am using spring-data-elasticsearch for a project to provide it with full text search functionality. We keep the real data in a relational database and relevant metadata along with respective id in elasticsearch. So for search results, only id field is required as the actual data will be retrieved from the relational database.

I am building the search query based on search criteria and then performing a queryForIds():

    SearchQuery searchQuery = new NativeSearchQueryBuilder()
            .withIndices(indexName)
            .withTypes(typeName)
            .withQuery(getQueryBuilder(searchParams))
            .withPageable(pageable)
            .build();

    return elasticsearchTemplate.queryForIds(searchQuery);

If I also need the total count for that specific searchQuery, I can do another elasticsearchTemplate.count(searchQuery) call, but that will be redundant as I understand. I think there is a way to get both the list of id and total count by using something like elasticsearchTemplate.queryForPage() in a single call.

Also, can I use a custom class in queryForPage(SearchQuery query, Class<T> clazz, SearchResultMapper mapper) which is not annotated with @Document? The actual document class is really big, and if I am not sure if passing large classes will cause any extra load on the engine since there are over 100 fields to be json mapped, but all I need is the id field. I will have a .withFields("id") in the query builder anyway.

How the edit got approved when the question is simply removed? I understand removing redundant text from the description (although not sure how a "thanks in advance" would hurt this stack overflow community so bad. I am re-editing my question title as the edit on that part was stupid, it removed vital information from the question. People will hardly get interested to read a damn "Count and Ids". — Zobayer Hasan, Feb 07 '16 at 16:01

score 8 · Accepted Answer · answered Feb 08 '16 at 16:54

If you want to prevent two calls to elasticsearch, i would suggest to write an custom ResultsExtractor:

SearchQuery searchQuery = new NativeSearchQueryBuilder().withIndices(indexName)
        .withTypes(typeName)
        .withQuery(queryBuilder)
        .withPageable(pageable)
        .build();
SearchResult result = template.query(searchQuery, new ResultsExtractor<SearchResult>() {

    @Override
    public SearchResult extract(SearchResponse response) {
        long totalHits = response.getHits()
                .totalHits();
        List<String> ids = new ArrayList<String>();
        for (SearchHit hit : response.getHits()) {
            if (hit != null) {
                ids.add(hit.getId());
            }
        }
        return new SearchResult(ids, totalHits);
    }
});
System.out.println(result.getIds());
System.out.println(result.getCount());

where SearchResult is a custom class:

public class SearchResult {

    List<String> ids;
    long count;

    //getter and setter
}

This way you can get the information you need from the elasticsearch SearchResponse.

Regarding your second question: As far as I can see, when calling queryForPage(SearchQuery query, Class<T> clazz, SearchResultMapper mapper) the passed class is not checked for the @Document annotation. Just try it out!

Interesting, as far as I remember I got error when I passed a class that does not have @Document annotation in it. But thanks for providing example about the extract() method, I solved my problem using this approach. — Zobayer Hasan, Feb 11 '16 at 06:13
I have another stack overflow question related to elasticsearch java api, In case you could help: [http://stackoverflow.com/questions/35323657/maintain-sequence-order-for-match-phrase-query-with-slop-in-elasticsearch-java-a](http://stackoverflow.com/questions/35323657/maintain-sequence-order-for-match-phrase-query-with-slop-in-elasticsearch-java-a) — Zobayer Hasan, Feb 11 '16 at 06:17

score 3 · Answer 2 · answered Sep 20 '18 at 04:18

One may also consider using AggregatedPage<T>. You can get the total number of records, total pages, current page records, etc. just like in Pageable<T>.

SearchQuery searchQuery = new NativeSearchQueryBuilder().withIndices(indexName)
        .withTypes(typeName)
        .withQuery(queryBuilder)
        .withPageable(pageable)
        .build();

AggregatedPage<ElasticDTO> queryResult = elasticsearchTemplate.queryForPage(searchQuery , ElasticDTO.class);

Getting count and list of ids using ElasticsearchTemplate in spring-data-elasticsearch

2 Answers2

Linked