Setting multiple custom analyzers to single field in elasticsearch

Question

I am working in ElasticSearch environment, I have installed elasticsearch on my local machine for version 5.4.3. I am trying to create index by defining some settings along with mappings. Following are my settings and mappings,

{  
   "settings":{  
      "index":{  
         "analysis":{  
            "analyzer":{  
               "index_analyzer":{  
                  "filter":[  
                     "standard",
                     "lowercase",
                     "asciifolding"
                  ],
                  "tokenizer":"standard"
               },
               "autocomplete":{  
                  "type":"custom",
                  "tokenizer":"standard",
                  "filter":[  
                     "lowercase",
                     "autocomplete_filter"
                  ]
               },
               "search_analyzer":{  
                  "filter":[  
                     "standard",
                     "lowercase",
                     "asciifolding"
                  ],
                  "tokenizer":"standard"
               },
               "sortable":{  
                  "filter":"lowercaseFilter",
                  "tokenizer":"keyword",
                  "type":"custom"
               }
            },
            "filter":{  
               "lowercaseFilter":{  
                  "type":"lowercase"
               },
               "autocomplete_filter":{  
                  "type":"edge_ngram",
                  "min_gram":1,
                  "max_gram":20
               }
            },
            "tokenizer":{  
               "keyword":{  
                  "type":"keyword"
               }
            }
         }
      }
   }
}

this is my mappings,

{  
   "geo_data":{  
      "_all":{  
         "enabled":true,
         "index_analyzer":"index_analyzer",
         "search_analyzer":"search_analyzer"
      },
      "properties":{  
         "subscriber_level":{  
            "analyzer":"index_analyzer,search_analyzer,autocomplete_analyzer",
            "type":"text"
         },
         "att_id":{  
            "analyzer":"index_analyzer,search_analyzer,autocomplete_analyzer",
            "type":"text"
         },
         "id":{  
            "include_in_all":false,
            "type":"text"
         },
         "name":{  
            "analyzer":"index_analyzer,search_analyzer,autocomplete_analyzer",
            "type":"text"
         },
         "state_name":{  
            "analyzer":"index_analyzer,search_analyzer,autocomplete_analyzer",
            "type":"text"
         }
      }
   }
}

What I want to achieve is, I want to apply all custom analyzers to a single field. But above mappings on fields for analyzers giving following exception,

{  
   "error":{  
      "root_cause":[  
         {  
            "type":"mapper_parsing_exception",
            "reason":"analyzer [index_analyzer,search_analyzer,autocomplete_analyzer] not found for field [subscriber_level]"
         }
      ],
      "type":"mapper_parsing_exception",
      "reason":"analyzer [index_analyzer,search_analyzer,autocomplete_analyzer] not found for field [subscriber_level]"
   },
   "status":400
}

Please anybody can help me to fix this issue, struggling on it.

score 6 · Accepted Answer · edited Jul 01 '17 at 10:05

you look to tokenize a same field with multiple analyzer. You can use multi-fields and apply different analyzer to each type inside multi-fields.

Also following this github issue, configuration for _all field are changed for 5.4. If your indexed is already exist,

PUT some_index/_mappings/type_name
{
            "_all": {
                "enabled": true,
                "type": "text",
                "analyzer": "index_analyzer"
            },
            "properties": {
                "subscriber_level": {
                    "type": "keyword",
                    "fields": {
                        "index_analyzed": {
                            "type": "text",
                            "analyzer": "index_analyzer"
                        },
                        "search_analyzed": {
                            "type": "text",
                            "analyzer": "search_analyzer"
                        },
                        "autocomplete_analyzed": {
                            "type": "text",
                            "analyzer": "autocomplete"
                        }
                    }
                },
                "att_id": {
                    "type": "keyword",
                    "fields": {
                        "index_analyzed": {
                            "type": "text",
                            "analyzer": "index_analyzer"
                        },
                        "search_analyzed": {
                            "type": "text",
                            "analyzer": "search_analyzer"
                        },
                        "autocomplete_analyzed": {
                            "type": "text",
                            "analyzer": "autocomplete"
                        }
                    }
                },
                "id": {
                    "include_in_all": false,
                    "type": "text"
                },
                "name": {
                    "type": "keyword",
                    "fields": {
                        "index_analyzed": {
                            "type": "text",
                            "analyzer": "index_analyzer"
                        },
                        "search_analyzed": {
                            "type": "text",
                            "analyzer": "search_analyzer"
                        },
                        "autocomplete_analyzed": {
                            "type": "text",
                            "analyzer": "autocomplete"
                        }
                    }
                },
                "state_name": {
                    "type": "keyword",
                    "fields": {
                        "index_analyzed": {
                            "type": "text",
                            "analyzer": "index_analyzer"
                        },
                        "search_analyzed": {
                            "type": "text",
                            "analyzer": "search_analyzer"
                        },
                        "autocomplete_analyzed": {
                            "type": "text",
                            "analyzer": "autocomplete"
                        }
                    }
            }
    },
    "settings": {
        "index": {
            "analysis": {
                "analyzer": {
                    "index_analyzer": {
                        "filter": [
                            "standard",
                            "lowercase",
                            "asciifolding"
                        ],
                        "tokenizer": "standard"
                    },
                    "autocomplete": {
                        "type": "custom",
                        "tokenizer": "standard",
                        "filter": [
                            "lowercase",
                            "autocomplete_filter"
                        ]
                    },
                    "search_analyzer": {
                        "filter": [
                            "standard",
                            "lowercase",
                            "asciifolding"
                        ],
                        "tokenizer": "standard"
                    },
                    "sortable": {
                        "filter": "lowercaseFilter",
                        "tokenizer": "keyword",
                        "type": "custom"
                    }
                },
                "filter": {
                    "lowercaseFilter": {
                        "type": "lowercase"
                    },
                    "autocomplete_filter": {
                        "type": "edge_ngram",
                        "min_gram": 1,
                        "max_gram": 20
                    }
                },
                "tokenizer": {
                    "keyword": {
                        "type": "keyword"
                    }
                }
            }
        }
    }
}

Now you use any of the analyzed field for query like following

POST some_index/_search
{
  "query": {
    "term": {
      "state_name.index_analyzed": {
        "value": "VALUE"
      }
    }
  }
}

Thanks

Thanks @users3775217 it worked for my scenario. Thank you very much. I have one doubt for ES, I am new to this ES world, but while exploring I observed that there are limited number of explanations and examples are available on the internet, even on stackoverflow limited number of users know about ES like you, even I also get to know about ES only just because our client wants it, I mean we can't find out any straight forward example for this for a new user, why is it so? It is a upcoming new framework to the market or it is old one. I am sorry if question is basic, I just have doubt. — Tushar Deshpande, Jul 01 '17 at 04:45
they have good examples to explain all needed concepts and parts. yeah i agree, how you using them is important. as always stack and github are good places to look for problems. — user3775217, Jul 01 '17 at 10:09

Setting multiple custom analyzers to single field in elasticsearch

1 Answers1