Stack Overflow
Stack Overflow

Questions tagged [elasticsearch-bulk-api]

For questions about the Elasticsearch bulk API.

For more information, see here: https://www.elastic.co/guide/en/elasticsearch/reference/current/docs-bulk.html

76 questions
79
votes
5 answers

How to use Bulk API to store the keywords in ES by using Python

I have to store some message in ElasticSearch integrate with my python program. Now what I try to store the message is: d={"message":"this is message"} for index_nr in range(1,5): ElasticSearchAPI.addToIndex(index_nr, d) print…
chengji18
  • 943
  • 1
  • 8
  • 9
28
votes
4 answers

BULK API : Malformed action/metadata line [3], expected START_OBJECT but found [VALUE_STRING]

Using Elasticsearch 5.5,getting the following error while posting this bulk request, unable to figure out what is wrong with the request. "type": "illegal_argument_exception", "reason": "Malformed action/metadata line [3], expected START_OBJECT but…
balaji thiruvengadam
  • 281
  • 1
  • 3
  • 3
23
votes
8 answers

What is the ideal bulk size formula in ElasticSearch?

I believe there should be a formula to calculate bulk indexing size in ElasticSearch. Probably followings are the variables of such a formula. Number of nodes Number of shards/index Document size RAM Disk write speed LAN speed I wonder If anyone…
shyos
  • 1,390
  • 1
  • 16
  • 29
19
votes
5 answers

Validation Failed: 1: no requests added in bulk indexing

I have a JSON file and I need to index it on ElasticSearch server. JSON file looks like this: { "sku": "1", "vbid": "1", "created": "Sun, 05 Oct 2014 03:35:58 +0000", "updated": "Sun, 06 Mar 2016 12:44:48 +0000", "type":…
rick
  • 1,675
  • 3
  • 15
  • 28
18
votes
1 answer

Bulk Update on ElasticSearch using NEST

I am trying to replacing the documents on ES using NEST. I am seeing the following options are available. Option #1: var documents = new List(); `var blkOperations = documents.Select(doc => new…
Sasi
  • 279
  • 1
  • 3
  • 12
13
votes
9 answers

How do I update multiple items in ElasticSearch?

Let's say I have a tag type in an ElasticSearch index, with the following mapping: { "tag": { "properties": { "tag": {"type": "string", "store": "yes"}, "aliases": {"type": "string"} } } } Each entry…
Doron
  • 3,176
  • 7
  • 35
  • 60
12
votes
4 answers

Reindexing Elastic search via Bulk API, scan and scroll

I am trying to re-index my Elastic search setup, currently looking at the Elastic search documentation and an example using the Python API I'm a little bit confused as to how this all works though. I was able to obtain the scroll ID from the Python…
Zack
  • 13,454
  • 24
  • 75
  • 113
8
votes
1 answer

Set Request Timeout in Elastic Search for bulk loads

I wanted to set the request time to 20 sec or more in Elasticsearch Bulk uploads. Default time is set to 10 sec and my Warning message days it takes 10.006 sec. And, right after displaying the waring the execution is throwing an error Now, I wanted…
Jack Daniel
  • 2,527
  • 3
  • 31
  • 52
6
votes
0 answers

How to import 1M records in batch with ElasticSearch rails?

I want to run a bulk import from my MySQL table to ES - for my model Wine - in my production server. There is 1.5M records. My model - code for ES gem: include Elasticsearch::Model include Elasticsearch::Model::Callbacks def…
alex.bour
  • 2,842
  • 9
  • 40
  • 66
6
votes
2 answers

elasticsearch python bulk api (elasticsearch-py)

I'm confused about py-elasticsearch bulk @Diolor solution works https://stackoverflow.com/questions/20288770/how-to-use-bulk-api-to-store-the-keywords-in-es-by-using-python, but I would like to use plain es.bulk() My code: from elasticsearch import…
sirkubax
  • 885
  • 2
  • 10
  • 19
4
votes
1 answer

Bulk Delete an attribute from a document in ElasticSearch 6+

I need to delete a particular attribute from all documents in ElasticSearch. Is there a way to do it and can it be done via Bulk API. I am unable to find an API to Bulk Delete an attribute from ElasticSearch. Is there any way to achieve the…
jatin mahajan
  • 81
  • 2
  • 7
4
votes
1 answer

Update nested field for millions of documents

I use bulk update with script in order to update a nested field, but this is very slow : POST…
eli0tt
  • 677
  • 1
  • 7
  • 19
4
votes
2 answers

Specify the _id field using Bulk.IndexMany in ElasticSearch

I'm facing a problem inserting document using bulk API (C# NEST v5.4). I've an array of documents and inside of the array I've my ID. My code is: documents = documents .ToArray(); Client.Bulk(bd => bd.IndexMany(documents, (descriptor, s) =>…
NatsuDragonEye
  • 109
  • 5
  • 18
3
votes
1 answer

Bulk API error while indexing data into elasticsearch

I want to import some data into elasticsearch using bulk API. this is the mapping I have created using Kibana dev tools: PUT /main-news-test-data { "mappings": { "properties": { "content": { "type": "text" }, "title":…
lydal
  • 802
  • 3
  • 15
  • 33
3
votes
1 answer

Elasticsearch Java API 7.0 Bulk Insert trying to basic example, throwing errors

Java 1.8, Elasticsearch Low and High Level Rest Clients 7.0.0 I am trying the simple example from the docs found here: Bulk API BulkRequest bulkRequest = new BulkRequest(); request.add(new IndexRequest("posts").id("1") …
Justin
  • 4,461
  • 22
  • 87
  • 152
1
2 3 4 5 6