1

Good morning,

Working on upgrading to the lastest version of the ELK stack and running into an issue with the field mapping. I have something similar to the following in my raw data:

{
  "_index" : "logstash-2016.04.21",
  "level1" : {
    "level2" : {
      "1" : "somevalue",
      "1.1" : "somevalue1"
    }
  }
}

And as we all know elasticsearch 2.X does not like dots (.) in field names, messes up the dot-notation. So as a total newbie to elasticsearch, I have not been able to find a way, or search the right terms, to find a way to fix this using mappings or analyzers or 'something i do not yet know about'. I am hoping the community here can help me find a way to fix this so I can re-index all of my current indexes as well as mapping it for future indexes.

Currently on ES version 1.7, running on debian

Thank you, Mike

Michael Ferrin
  • 47
  • 8
  • While the logstash plugin might work. I think in my particular situation it isn't. I am trying to go use stream2es to copy from one instance to another. Is there anything directly with elasticsearch (plugin, mapping, template) that might be able to do this? – Michael Ferrin Apr 21 '16 at 20:39

1 Answers1

2

If you can't fix the input, logstash has a de_dot filter for this purpose.

Alain Collins
  • 16,268
  • 2
  • 32
  • 55
  • Thanks @AlainCollins! Pardon my newbie-ness but after installing the plugin, where would I include this to get it to remove the dots? – Michael Ferrin Apr 21 '16 at 16:25
  • Also do you know of a way to use this to reindex existing indexes? – Michael Ferrin Apr 21 '16 at 16:27
  • de_dot is a filter that will, by default, run through all of your fields and remove the dots. You can configure it to look at some fields. – Alain Collins Apr 21 '16 at 17:15
  • How to use the de_dot filter if I am not using log stash? I want to re-index to have a new index with all dots in the fields replaced by an underscore – yancheelo Aug 29 '22 at 08:56
  • @yancheelo try this answer from another contributor? https://stackoverflow.com/questions/43120430/elasticsearch-mapping-rename-existing-field – Alain Collins Sep 23 '22 at 17:50