Stack Overflow
Stack Overflow

Questions tagged [elasticsearch-bulk]

16 questions
62
votes
3 answers

Elasticsearch Bulk API - Index vs Create/Update

I'm using the Elasticsearch Bulk API to create or update documents. I do actually know if they are creates or updates, but I can simplify my code by just making them all index, or "upserts" in the SQL sense. Is there any disadvantage in using…
Kong
  • 8,792
  • 15
  • 68
  • 98
9
votes
0 answers

ElasticSearch error ""Message":"Request size exceeded 10485760 bytes"

While uploading a 50 mb size huge JSON string in ElasticSearch using this method - public static void postData(String json, String index, String type) { RestClient client = RestClient.builder(new HttpHost(testHostPreProd, 443,…
user9755712
2
votes
3 answers

elasticsearch bulk indexing and redundant data in action part

When indexing data using bulk API of elasticsearch here is the sample json from the site documentation POST _bulk { "index" : { "_index" : "test", "_type" : "_doc", "_id" : "1" } } { "field1" : "value1" } { "index" : { "_index" : "test", "_type" :…
Waku-2
  • 1,136
  • 2
  • 13
  • 26
2
votes
0 answers

How to do Elasticsearch Bulk Insert in Python

I have a JSON array which contains a list of data that needs to be pushed to an ElasticSearch instance. Something like [ {"Message": "2aTeFCKTYWwfrF"}, {"Message": "2aTeFCKTYWwfrD"}, {"Message": "2aTeFCKTYWwfrW"}.......{"Message":…
tsaebeht
  • 1,570
  • 5
  • 18
  • 32
1
vote
0 answers

Elasticsearch : UpdateByQuery API Response returns wrong status

I am facing issue with UpdateByQuery API while trying to update a document which doesn’t exist in Elastic search Problem description We are creating one index for each day like test_index-2020.03.11, test_index-2020.03.12… and we maintain eight…
Om S patel
  • 31
  • 2
1
vote
1 answer

How to insert an already created json-format string to Elasticsearch Bulk

In a python script, I'm trying for elasticsearch.helpers.bulk to store multiple records. I will get a json-format string from another software, and I want to attach it in the source part I got the helpers.bulk format by this answer part of my…
haeny
  • 363
  • 2
  • 5
  • 15
1
vote
1 answer

ElasticSearch is unable to recognize Context-Type header with encoding defined

I've spent some time trying to fix the elastic search bulk upload warning: Content type detection for rest requests is deprecated. Specify the content type using the [Content-Type] header My request is below: POST http://elasticserver/_bulk…
Mando
  • 11,414
  • 17
  • 86
  • 167
1
vote
0 answers

Optimizing Bulk Indexing in elasticsearch

We have an elastic search cluster of 3 nodes of the following configurations #Cpu Cores Memory(GB) Disk(GB) IO Performance 36 244.0 48000 very high The machines are in 3 different zones namely…
STandon
  • 76
  • 7
0
votes
2 answers

Partially updating elasticsearch list field value using python

The purpose of this question is to ask the community how to go about partially updating a field without removing any other contents of that field. There are many examples in StackOverflow to partially update ElasticSearch _source fields using…
Jenobi
  • 368
  • 4
  • 12
0
votes
1 answer

Efficient Bulk Loading Options for Elasticsearch in Python

I am trying to ingest a large amount of data into Elasticsearch using Python. For this purpose, I am using the bulk API helper and I have developed a function that looks something like this, def __load(self, docs, index): try: # begin…
Minura Punchihewa
  • 1,498
  • 1
  • 12
  • 35
0
votes
1 answer

elasticsearch bulk insert exception while uploading

I am getting an exception when trying to bulk insert into elasticsearch v6.6.0. It was fine earlier. Here's the exception message: Elasticsearch bulk insert exception, TransportError(503, 'circuit_breaking_exception', '[parent] Data too large, data…
Azima
  • 3,835
  • 15
  • 49
  • 95
0
votes
1 answer

Insert header for each document before uploading to elastic search

I have a ndjson file with the below format {"field1": "data1" , "field2": "data2"} {"field1": "data1" , "field2": "data2"} .... I want to add a header like {"index": {}} before each document before using the bulk operation I found a similar…
Abhijeet Ramgir
  • 51
  • 1
  • 2
  • 8
0
votes
1 answer

upserting batches into elasticsearch store with bulk API

I have huge set of documents with same index and same type but obviously different ids. I want to either update existing ones or insert new in batches. How can I achieve it using bulk indexing API? I want to do something like below but it throws…
Rohanil
  • 1,717
  • 5
  • 22
  • 47
0
votes
1 answer

Elasticsearch bulk API returning 400 error in Java

I tried a lot of things to check why is it showing a malformed request. Refer to this question to properly format my request body. Parent specification in elasticsearch for first node Here is my code. I am using Unirest API to send requests. In the…
Amriteya
  • 1,118
  • 15
  • 37
0
votes
1 answer

Parent specification in elasticsearch for first node

I am using the _bulk api for indexing my data. I have a parent field for every document except the first document. Normally my documents will have the following meta-data: {"index":{"_id":"11111", "parent": "00000"}} What should I specify in the…
Amriteya
  • 1,118
  • 15
  • 37
1
2