How do I update multiple items in ElasticSearch?

Question

Let's say I have a tag type in an ElasticSearch index, with the following mapping:

{
    "tag": {
        "properties": {
            "tag": {"type": "string", "store": "yes"},
            "aliases": {"type": "string"}
        }
    }
}

Each entry is a tag, and an array of aliases to that tag. Here is an example item:

{
    "word": "weak",
    "aliases": ["anemic", "anaemic", "faint", "flimsy"]
}

From time to time, I want to add new tag words with their aliases, and add new aliases to existing tag words.

Adding new tag words with their aliases is easy, it's just a new Document. However, how can I add new aliases to existing tag words in a sane way?

I know I can just search for the tag word, get its document, search to see if the alias already exists in the array of aliases, if not add it, than save. However - this does not sound like a good solution.

Is there a way to do a bulk update?

score 12 · Answer 1 · edited Nov 02 '15 at 03:32

12

Try this using _bulk:

http://127.0.0.1:9200/myindex/type/_bulk
{
"update": {
    "_index": "myindex",
    "_type": "type",
    "_id": "myid"
}
}{
"doc": {
    "field": "new value"
}
}{
"update": {
    "_index": "myindex",
    "_type": "type",
    "_id": "id"
}
}{
"doc": {
    "field": "new value"
}
}

edited Nov 02 '15 at 03:32

Chris Martin

30,334
10
78
137

answered Jun 08 '15 at 11:16

Varshaan

555
9
22

Thanks for the example. It's not need to specify `_index` and `_type` fields in every `update: {}` object, as it is already specified in request URL: `http://127.0.0.1:9200/myindex/type/_bulk` – chill appreciator May 02 '20 at 20:50
To make this example working we need to write "object per line": `{"update": {"_id": "myId0"}} {"doc": {"field": "new value 0"}} {"update": {"_id": "myId1}} {"doc": {"field": "new value 1"}} ` Also, make sure you have added `Content-Type: application/x-ndjson` header to the request. – chill appreciator May 02 '20 at 20:59
@Varshaan: You have made my day!! Thanks Buddy!! :) – shivshankar Oct 11 '20 at 15:46

score 11 · Answer 2 · edited Jan 01 '19 at 17:05

11

All updates in ElasticSearch are done by finding the record, deleting the old version and adding the new version. You can save a little bit on moving records all the way to the client by using Update API. It would still require finding the record though.

What you, probably, want is Update by query.

edited Jan 01 '19 at 17:05

Jay Stevens

5,863
9
44
67

answered Apr 17 '12 at 13:16

imotov

28,277
3
90
82

1

Update by query is still not added to Elastic, but a [plugin](https://github.com/yakaz/elasticsearch-action-updatebyquery/) exists. – Chris Martin Nov 02 '15 at 03:30
1

As of Elasticsearch 2.3, update-by-query is available - https://www.elastic.co/guide/en/elasticsearch/reference/2.3/docs-update-by-query.html – PhaedrusTheGreek May 06 '16 at 17:04

score 4 · Answer 3 · edited Jan 25 '17 at 16:54

4

This works for me.

input_list.dat:

{ "index" : { "_index": "my_index", "_type": "my_type", "_id": "existing-value" } }

{ "Field_to_update": "New_Value" }

{ "index" : { "_index": "my_index", "_type": "my_type", "_id": "existing_value" } }

{ "Field_to_update": "New_Value" }

Command:

curl -k -XPOST 'https://my_host:9200/my_url/_bulk' --data-binary "@input_list.dat"; echo

edited Jan 25 '17 at 16:54

Debosmit Ray

5,228
2
27
43

answered Jan 21 '16 at 01:50

Rockoder

746
2
11
22

score 4 · Answer 4 · answered May 06 '16 at 17:23

Elasticsearch 2.3.0 introduced the Update By Query API as part of the long awaited Reindex API.

As an example, here is how you could update all documents to delete a certain field if it exists:

POST /myindex/mytype/_update_by_query
{
  "script": {
    "inline": "ctx._source.remove(\"remove\")"
  },
  "query": {
    "exists": {
      "field": "remove"
    }
  }
}

The above example uses inline scripting, so be sure to enable it in elasticsearch.yml with script.inline: on.

score 2 · Answer 5 · edited Nov 02 '15 at 03:36

2

Elastic Search has a Update API. With that API you can do the following:

curl -XPOST 'localhost:9200/test/tag/weak/_update' -d '{
    "script" : "ctx._source.aliases += faint"
}'

edited Nov 02 '15 at 03:36

Chris Martin

30,334
10
78
137

answered Apr 09 '13 at 15:50

Eric Vidal

29
3

Can you please tell me how can I implement update a single record in php? – Umair Hamid Nov 07 '14 at 06:51

score 1 · Answer 6 · answered Mar 09 '20 at 06:15

You can use the ElasticSeach Bulk API for updating the multiple documents using a single API call

CURL example

curl --location --request POST 'localhost:9200/whatsapp/_bulk' \
--header 'Content-Type: application/json' \
--data-raw '{ "update" : {"_id" : 692, "_index" : "whatsapp","_type":"_doc","retry_on_conflict" : 3} }
{ "doc" : {"thread_status" : 1} }
{ "update" : {"_id" : 693, "_index" : "whatsapp","_type":"_doc","retry_on_conflict" : 3} }
{ "doc" : {"thread_status" : 1} }

'

NOTE The final line of data must end with a newline character \n. That' why you will notice ' at the end line of json.

score 0 · Answer 7 · answered Apr 01 '14 at 14:04

0

Also if you add same value with same id, it will automatically update older data.

answered Apr 01 '14 at 14:04

Mehmet Uyarovic

304
2
9

Update or replace? – bfontaine Mar 03 '23 at 13:54

score 0 · Answer 8 · edited Nov 02 '15 at 03:32

Elasticsearch's bulk APIs can be used for update requests as well, at least for the Java client.

List list = new Arraylist();
list.add("hello");
BulkProcessor bulk = new BulkProcessor();
UpdateRequest update = new UpdateRequest("index", "type", "id1");
update.script("ctx._source.aliases+= newaliases");  //dynamic script
update.addScriptParam("newaliases", list);
bulk.add(update);

Note that dynamic scripting is disabled in newer versions of elasticsearch. Either enable that or use pre-compiled scripts for using this feature.

score 0 · Answer 9 · edited Nov 10 '15 at 08:48

You can do the same using spring java client using the following code. Following are the dependencies used in the code.

import org.elasticsearch.action.update.UpdateRequest;

import org.elasticsearch.index.query.QueryBuilder;

import org.springframework.data.elasticsearch.core.query.UpdateQuery;

import org.springframework.data.elasticsearch.core.query.UpdateQueryBuilder;

private UpdateQuery updateExistingDocument(String Id) {
    // Add updatedDateTime, CreatedDateTime, CreateBy, UpdatedBy field in existing documents in Elastic Search Engine
    UpdateRequest updateRequest = new UpdateRequest().doc("UpdatedDateTime", new Date(), "CreatedDateTime", new Date(), "CreatedBy", "admin", "UpdatedBy", "admin");

    // Create updateQuery
    UpdateQuery updateQuery = new UpdateQueryBuilder().withId(Id).withClass(ElasticSearchDocument.class).build();
    updateQuery.setUpdateRequest(updateRequest);

    // Execute update
     elasticsearchTemplate.update(updateQuery);
}

How do I update multiple items in ElasticSearch?

9 Answers9