2

I found great example on helpers.bulk function but I cannot find a response code. Everything I get from

helpers.bulk(es, actions)

is

(1, [])
Pavol Travnik
  • 853
  • 3
  • 14
  • 32
  • Can You show us your complete method? – NeoVe Sep 24 '17 at 16:42
  • It is the same as presented in the [example](https://stackoverflow.com/questions/20288770/how-to-use-bulk-api-to-store-the-keywords-in-es-by-using-python). I expected, that `helpers.bulk(es, actions)` returns something like `Response [200]` or `{"acknowledged":true}` – Pavol Travnik Sep 24 '17 at 18:11
  • I am seeing the same issue, were you able to get the response? – user 923227 Oct 31 '18 at 00:42
  • What if we want to get the same response as when the bulk API is called from command line? – elachell Feb 10 '20 at 20:17

2 Answers2

2

As written in documentation:

It returns a tuple with summary information - number of successfully executed actions and either list of errors or number of errors if stats_only is set to True [...] If you need to process a lot of data and want to ignore/collect errors please consider using the streaming_bulk() helper which will just return the errors and not store them in memory.

with streaming_bulk() you have to use raise_on_error parameter for raise on error. if you want to collect a lot of data i suggest to use parallel_bulk() that is faster and more intuitive

Lupanoide
  • 3,132
  • 20
  • 36
1

I am doing the same way on AWS Lambda to push data to ElasticSearch:

print("Pushing data to ES <:", ESindex, ESmapping, ">", helpers.bulk(g.esClient, dataGenerator()))

Here the response 

(1, [])

means 1 record pushed successfully to Elasticsearch. In case of a failure I got an error log like below and the Lambda Function Failed. (I had removed the [] array brackets from the Json Data.)

RequestError(400, 'action_request_validation_exception', 'Validation Failed: 1: index is missing;2: type is missing;3: index is missing;4: type is missing;5: index is missing;6: type is missing;7: index is missing;8: type is missing;9: index is missing;10: type is missing;11: index is missing;12: type is missing;13: index is missing;14: type is missing;15: index is missing;16: type is missing;17: index is missing;18: type is missing;19: index is missing;20: type is missing;21: index is missing;22: type is missing;23: index is missing;24: type is missing;25: index is missing;26: type is missing;27: index is missing;28: type is missing;29: index is missing;30: type is missing;'): RequestError
user 923227
  • 2,528
  • 4
  • 27
  • 46