3

Is there any way to get only failed documents in the response while bulk request using Elasticsearch Java High-Level REST Client.

Currently, ES is sending all the succeed and failed document in the response and we are reprocessing all the failed document, we are iterating BulkItemResponse to find the failed document and reprocessing it.

private BulkRequest createBulkRequestsForRetry(BulkResponse bulkItemResponses, BulkRequest currentBulkRequest) {
        BulkRequest bulkRequest = new BulkRequest();
        int index = 0;
        for (BulkItemResponse bulkItemResponse : bulkItemResponses.getItems()) {
            if (bulkItemResponse.isFailed()) {
                bulkRequest.add(currentBulkRequest.requests().get(index));
            }
            index++;
        }
        return bulkRequest;
    }

As bulkItemResponse.getItems() representing each action performed in the bulk operation (in the same order!).

pramesh
  • 501
  • 3
  • 11

1 Answers1

2

As of now even in the latest 7.7 version of the client, this feature is not present, if you want this feature you can create an issue or better submit a PR addressing the issue, it would be a very useful feature to have.

You can have a look at the complete BulkResponse and see yourself there is no such functionality present in the master branch as well.

Also here you can create an issue to Elasticsearch.

Amit
  • 30,756
  • 6
  • 57
  • 88
  • 1
    Agreed. Doc says "To return only information about failed operations, use the `filter_path` query parameter with an argument of `items.*.error.`" but this applies only to the REST API as this filter_path option is not exposed in the REST Client. See also https://github.com/elastic/elasticsearch/issues/53846 – dadoonet Jun 05 '20 at 08:21
  • @dadoonet wow great to hear from Developer Evangelist at Elastic :) and thanks for providing the link for an existing issue, went through and quite useful why it was not implemented. – Amit Jun 05 '20 at 08:26
  • 1
    I just added my thoughts to the issue :) – dadoonet Jun 05 '20 at 08:27
  • @dadoonet you added a lot more clarity. – Amit Jun 05 '20 at 08:30