ElasticSearch search results with variable values

Question

I'm looking to use elasticsearch for a search bar with autocompletion feature. I have a set of questions and they have multiples variables that I'd like to substitute with other values.

A template of a question would be: Do you like FIELDS? with FIELDS replaced by math, physics, history and then when querying the question "do you like", it would show multiple hits:

• do you like math?
• do you like physics?
• do you like history?

I saw elasticseach as a synonyms analyzer and thought it could be used for this use case but it doesn't seem to work like I expected. Here's below what I have so far.

create the index

{
        "mappings": {
            "properties": {
                "my_field": {
                    "type": "search_as_you_type",
                    "analyzer": "standard",
                    "search_analyzer": "synonym_analyzer"
                }
            }
        },
        "settings": {
            "index": {
                "analysis": {
                    "analyzer": {
                        "synonym_analyzer": {
                            "tokenizer": "whitespace",
                            "filter": ["my_synonyms"]
                        }
                    },
                    "filter": {
                        "my_synonyms": {
                            "type": "synonym",
                            "synonyms": [
                                "FIELDS => math, physics, history"
                            ]
                        }
                    }
                }
            }
        }
    }

query

{
        "query": {
            "multi_match": {
                "query": partial_question,
                "type": "bool_prefix",
                "fields": [
                    "my_field",
                    "my_field._2gram",
                    "my_field._3gram",
                    "my_field._index_prefix"
                ]
            }
        }
    }

The result is one result "Do you like FIELDS?"

Answer 1

@hansley answer would work but as wildcard queries are costly, you could simply use the prefix query without changing anything in your index.

Although there are various ways to implement Autosuggest in ES and considering its importance and popularity I've written a detailed blog on various approaches and their trade-off Also my this SO answer could give you information on functional and non-function requirements to build an Autosuggest feature.

End to end example using prefix query:

Default index mapping, which creates a .keyword field for each text field:

Index sample docs:

{
  "title" : "i like red car"
}

{
  "title" : "do you like math?"
}

{
  "title" : "do you like physics?"
}

Search query

{
  "query": {
    "prefix": {
      "title.keyword": {
        "value": "do you like"
      }
    }
  }
}

Search results

"hits": [
      {
        "_index": "partialpre",
        "_type": "_doc",
        "_id": "1",
        "_score": 1.0,
        "_source": {
          "title": "do you like math?"
        }
      },
      {
        "_index": "partialpre",
        "_type": "_doc",
        "_id": "2",
        "_score": 1.0,
        "_source": {
          "title": "do you like physics?"
        }
      }
    ]

Answer 2

If I understood your issue well, I did a suggested answers which you may use. I used the wildcard query alongside a normalizer to lowercase all the values:

Here is the mapping of my index:

PUT multiple-fields
{
  "settings": {
    "analysis": {
      "normalizer": {
        "lowercase_normalizer": {
          "type":"custom",
        "filter": ["lowercase"]
        }
        
      }
    }
  },
  "mappings": {
    "properties": {
      "quest":{
        "type": "keyword",
        "normalizer": "lowercase_normalizer"
      }
    }
  }
}

I ingested the following data in the index:

"quest":"do you like math?"

"quest":"do you like physics?"

"quest":"do you like history?"

"quest":"I think i do like you"

"quest":"I like red car"

"quest":"do you not like"

"quest":"Do you like math?"

Based on the values, I created the following query:

GET multiple-fields/_search
{
  "query": {
    "wildcard": {
      "quest": {
        "value": "do you like*"
      }
    }
  }
}

The response was:

"hits" : [
  {
    "_index" : "multiple-fields",
    "_type" : "_doc",
    "_id" : "bue1e3QBsTCl1BZvB0by",
    "_score" : 1.0,
    "_source" : {
      "quest" : "do you like math?"
    }
  },
  {
    "_index" : "multiple-fields",
    "_type" : "_doc",
    "_id" : "cOe1e3QBsTCl1BZvD0Yh",
    "_score" : 1.0,
    "_source" : {
      "quest" : "do you like physics?"
    }
  },
  {
    "_index" : "multiple-fields",
    "_type" : "_doc",
    "_id" : "cee1e3QBsTCl1BZvE0Zq",
    "_score" : 1.0,
    "_source" : {
      "quest" : "do you like history?"
    }
  },
  {
    "_index" : "multiple-fields",
    "_type" : "_doc",
    "_id" : "2-e1e3QBsTCl1BZvLUak",
    "_score" : 1.0,
    "_source" : {
      "quest" : "Do you like math?"
    }
  }
]

Links: https://www.elastic.co/guide/en/elasticsearch/reference/current/normalizer.html https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl-wildcard-query.html

Note: However, it might affect the performance if you are using a wildcard

Let me know if it would help you, else we can work on another solution thank you.

Answer 3

I think you are looking in the wrong direction with the synonym. What you want is a suggester. You will have a little specific work to do but you will be able to create very powerful suggestions for your uses cases.