2

I have several old indices which I'd like to change mapping of. Now I need to transform old mapping into new ones. I've been reading on stackoverflow like

This one

This one

and this one

But I don't think they're the exact answer to my question.

Many of those use a different index name, so operation is easier that way, but due to the setup, we use a more crude way of curl-ing the data, which requires the exact index name.

Now my question is: what's a good way to reindex an old index (or several indices) to new one

ex.

my current address would be 'http://address-to-server:port/cluster-name/index-name'

I want to be able to reindex the old data so that when i curl -XGET 'http://address-to-server:port/cluster-name/index-name' it'll give the the old data in new mapping (some additional fields, some modified)

Community
  • 1
  • 1
JChao
  • 2,178
  • 5
  • 35
  • 65
  • You can use: https://www.npmjs.com/package/elasticdump – Nir Alfasi Feb 05 '16 at 20:10
  • but will this be able to handle a modification of fields? Say in old index mapping I have `field1 ` and the new mapping will have `field1`. Will it cause any error? – JChao Feb 05 '16 at 22:23
  • This is an import/export tool. If your mapping to data is broken - no tool can fix it... – Nir Alfasi Feb 05 '16 at 22:44

3 Answers3

1

As my previous answer you can use reindexing plugins to perform the mapping change actions

https://github.com/codelibs/elasticsearch-reindexing

And also make sure that your new mapping is valid and will adopt your old data by checking sample input data

Community
  • 1
  • 1
Pandiyan Cool
  • 6,381
  • 8
  • 51
  • 87
1

Following method can be used to re-index the data.

  1. Transfer data from original index to some dummy index. You can use below command

    elasticsearch-reindex -f "http://localhost:9200/original_index/type/" -t "http://localhost:9200/dummy_index/type/"

    This command is to be used in terminal.

  2. Delete the original index. DELETE /original_index

  3. Create the index with same name as original index and with changed mapping.

  4. Move the data back from dummy index to original index

    elasticsearch-reindex -f "http://localhost:9200/dummy_index/type/" -t "http://localhost:9200/original_index/type/"

THis way you can restore your data with new mappings.

NOTE: You need to install npm plugin for this.

Read this to install elasticsearch-reindex command.

Hope this helps.

Richa
  • 7,419
  • 6
  • 25
  • 34
  • what if I created the mapping just before the with reindex after delete the original index, should I have to use the again create desire mapping to back data from dummy index to original index?? – Indrajeet Gour Apr 16 '19 at 05:27
  • 1
    Instead of step 3 and 4, just create an alias (with the original index name) to the new index. So no need to move data back. – JVS Jul 08 '20 at 05:48
  • This is very hard to pull off in production, what if after the transfer of data to the `dummy_index` is finished, there is a new data in the `original_index`. This data is now lost forever. – DollarAkshay May 22 '21 at 23:53
0

For those who are looking to run reindex script to update the same index. Basically "reindexing to the same index" is an "update by query" (https://www.elastic.co/guide/en/elasticsearch/reference/current/docs-update-by-query.html). You can use the same script clause as with reindex and apply it to a query result

serejja
  • 22,901
  • 6
  • 64
  • 72
  • at the time when I asked this, our version of ES was still on 1.x, which doesn't have update-by-query or delete-by-query functions. Gotta say, adding these functions make things a lot simpler – JChao Jun 15 '18 at 21:23
  • 1
    It is not the same. You can't change the _id using update by query. – lilezek Nov 22 '18 at 15:37
  • Update by query will not allow you to change existing mappings. For that you need a new index. – Marc Oct 30 '20 at 09:09