Elasticsearch manipulate existing field value to add new field

Question

I try to add new field which is value comes from hashed existing field value. So, i want to do; my_index.hashedusername(new field) = crc32(my_index.username) (existing field)

For example

POST  _update_by_query
{
  "query": {
    "match_all": {}
  },
  "script" : {
      "source": "ctx._source.hashedusername = crc32(ctx._source.username);"
  }
}

Please give me an idea how to do this..

Do you want to update existing docs or are you looking for a solution to add that hashed field upon ingestion? — Joe - GMapsBook.com, Nov 10 '20 at 11:33
I want to update existing docs with new field (hashedusername) — data_m, Nov 10 '20 at 11:38

score 1 · Accepted Answer · answered Nov 10 '20 at 12:11

1

java.util.zip.CRC32 is not available in the shared painless API so mocking that package will be non-trivial -- perhaps even unreasonable.

I'd suggest to compute the CRC32 hashes beforehand and only then send the docs to ES. Alternatively, scroll through all your documents, compute the hash and bulk-update your documents.

The painless API was designed to perform comparatively simple tasks and CRC32 is certainly outside of its purpose.

answered Nov 10 '20 at 12:11

Joe - GMapsBook.com

15,787
4
23
68

Thanks for your reply. I have 400 M document. So manually change is hard for me. Do you know any other way to add new field which is hold unique hash code? – data_m Nov 10 '20 at 12:19
1

No prob. Perhaps this would help? https://stackoverflow.com/q/2624192/8160318 – Joe - GMapsBook.com Nov 10 '20 at 12:25

Elasticsearch manipulate existing field value to add new field

1 Answers1