RestHighlevelClient Sliced Scroll not working

Question

I am trying to fetch a large data records from Elasticsearch (Version - 6.4.2) in a minimal time. To do that, I am using multi threading to run each slice scroll in a separate thread.

example: My query has 3 slices, so:

thread 1 - slice id 0
thread 2 - slice id 1
thread 3 - slice id 2

But while trying to get the results from slice id 1 and 2 it's returning 0 hits.

I am trying RestHighLevelClient#search()

public SearchResponse GetResponse(Integer sliceCount, Integer totalSlice, Integer searchSize, String[] identifiers ) throws IOException 
{

    SliceBuilder sliceBuilder = new SliceBuilder(sliceCount, totalSlice);
    SearchSourceBuilder slicedSource = new SearchSourceBuilder().slice(sliceBuilder);
    slicedSource = slicedSource
        .query(QueryBuilders.boolQuery()
        .must(QueryBuilders.termsQuery("_id", identifiers)));
    return client.search(
      new SearchRequest()
        .indices("indexname")
        .types("type")
        .scroll("scrolltime")
        .source(slicedSource)
      );

}

What’s your scrolltime? Is it enough to get the request executed? — Mysterion, Jan 16 '19 at 10:10
Scroll time is 2m and yes terms query is working as slice id 0 is returning all the results. The issue is that its not getting sliced so that I can fetching them in parallel to reduce the overall time. — Pramod Agrawal, Jan 16 '19 at 18:22

score 0 · Answer 1 · answered Jun 30 '20 at 21:53

0

You need to reuse SearchSourceBuilder rather to create it for each thread.

Please refer: https://github.com/elastic/elasticsearch/issues/24766

answered Jun 30 '20 at 21:53

kamini patel

1

RestHighlevelClient Sliced Scroll not working

1 Answers1