Why is there a difference in the search results when querying elasticsearch using a term query?

Question

I have recently started learning elasticsearch and I am getting a difference in the search results of my query. The mapping of the index named "products" is provided below(I am pasting the response from my Kibana console tool) :

    {
    "products" : {
"mappings" : {
  "properties" : {
    "in_stock" : {
      "type" : "long"
    },
    "name" : {
      "type" : "text",
      "fields" : {
        "keyword" : {
          "type" : "keyword",
          "ignore_above" : 256
        }
      }
    },
    "price" : {
      "type" : "long"
    },
    "tags" : {
      "type" : "text",
      "fields" : {
        "keyword" : {
          "type" : "keyword",
          "ignore_above" : 256
        }
      }
    }
  }
}
}
}

The data in the index is as follows(I am pasting the response from my Kibana console tool):

{
   "took" : 0,
  "timed_out" : false,
  "_shards" : {
  "total" : 1,
  "successful" : 1,
  "skipped" : 0,
  "failed" : 0
  },
  "hits" : {
"total" : {
  "value" : 16,
  "relation" : "eq"
},
"max_score" : 1.0,
"hits" : [
  {
    "_index" : "products",
    "_type" : "_doc",
    "_id" : "202",
    "_score" : 1.0,
    "_source" : {
      "name" : "Vegetable Chopper",
      "price" : 10,
      "in_stock" : 250,
      "tags" : [
        "kitchen appliances",
        "vegetable slicer",
        "chopper"
      ]
    }
  },
  {
    "_index" : "products",
    "_type" : "_doc",
    "_id" : "203",
    "_score" : 1.0,
    "_source" : {
      "name" : "Dish Washer",
      "price" : 90,
      "in_stock" : 60,
      "tags" : [
        "kitchen appliances",
        "electrical",
        "electric washer"
      ]
    }
  },
  {
    "_index" : "products",
    "_type" : "_doc",
    "_id" : "205",
    "_score" : 1.0,
    "_source" : {
      "name" : "Microwave Oven",
      "price" : 100,
      "in_stock" : 50,
      "tags" : [
        "kitchen appliances",
        "electricals",
        "oven",
        "oven toaster",
        "microwave"
      ]
    }
  },
  {
    "_index" : "products",
    "_type" : "_doc",
    "_id" : "206",
    "_score" : 1.0,
    "_source" : {
      "name" : "Mixer Grinder",
      "price" : 55,
      "in_stock" : 130,
      "tags" : [
        "kitchen appliances",
        "electricals",
        "mixer",
        "grinder",
        "juicer",
        "food processor"
      ]
    }
  },
  {
    "_index" : "products",
    "_type" : "_doc",
    "_id" : "207",
    "_score" : 1.0,
    "_source" : {
      "name" : "Fruit Juicer",
      "price" : 40,
      "in_stock" : 100,
      "tags" : [
        "kitchen appliances",
        "electicals",
        "juicer",
        "mixer",
        "electric juicer",
        "juice maker"
      ]
    }
  },
  {
    "_index" : "products",
    "_type" : "_doc",
    "_id" : "208",
    "_score" : 1.0,
    "_source" : {
      "name" : "Knife Set",
      "price" : 15,
      "in_stock" : 250,
      "tags" : [
        "kitchen knife",
        "steel knives",
        "cutlery"
      ]
    }
  },
  {
    "_index" : "products",
    "_type" : "_doc",
    "_id" : "209",
    "_score" : 1.0,
    "_source" : {
      "name" : "Rice Maker",
      "price" : 85,
      "in_stock" : 60,
      "tags" : [
        "kitchen appliances",
        "electricals",
        "electric rice cooker",
        "electric pressure cooker"
      ]
    }
  },
  {
    "_index" : "products",
    "_type" : "_doc",
    "_id" : "210",
    "_score" : 1.0,
    "_source" : {
      "name" : "Induction Cooktop",
      "price" : 30,
      "in_stock" : 150,
      "tags" : [
        "kitchen appliances",
        "electricals",
        "hot plate heater",
        "electric hot place",
        "induction cooker",
        "induction stove"
      ]
    }
  },
  {
    "_index" : "products",
    "_type" : "_doc",
    "_id" : "211",
    "_score" : 1.0,
    "_source" : {
      "name" : "Coffee Maker",
      "price" : 50,
      "in_stock" : 100,
      "tags" : [
        "kitchen appliances",
        "electricals"
      ]
    }
  },
  {
    "_index" : "products",
    "_type" : "_doc",
    "_id" : "212",
    "_score" : 1.0,
    "_source" : {
      "name" : "Wine Glasses Set",
      "price" : 50,
      "in_stock" : 70,
      "tags" : [
        "kitchen and dining",
        "glassware",
        "stemware"
      ]
    }
  },
  {
    "_index" : "products",
    "_type" : "_doc",
    "_id" : "213",
    "_score" : 1.0,
    "_source" : {
      "name" : "Dinner Set",
      "price" : 100,
      "in_stock" : 40,
      "tags" : [
        "kitchen and dining",
        "crockery",
        "full dinner set"
      ]
    }
  },
  {
    "_index" : "products",
    "_type" : "_doc",
    "_id" : "214",
    "_score" : 1.0,
    "_source" : {
      "name" : "Whiskey Glasses Set",
      "price" : 60,
      "in_stock" : 50,
      "tags" : [
        "kitchen and dining",
        "glassware",
        "whiskey glasses",
        "old fashioned glass",
        "rocks glass",
        "short tumbler"
      ]
    }
  },
  {
    "_index" : "products",
    "_type" : "_doc",
    "_id" : "215",
    "_score" : 1.0,
    "_source" : {
      "name" : "Mug And Saucer Set",
      "price" : 35,
      "in_stock" : 60,
      "tags" : [
        "kitchen and dining",
        "mug set",
        "mugs and saucer",
        "crockery set"
      ]
    }
  },
  {
    "_index" : "products",
    "_type" : "_doc",
    "_id" : "201",
    "_score" : 1.0,
    "_source" : {
      "name" : "Milk Frother",
      "price" : 25,
      "in_stock" : 15,
      "tags" : [
        "kitchen appliances",
        "electricals",
        "milk"
      ]
    }
  },
  {
    "_index" : "products",
    "_type" : "_doc",
    "_id" : "200",
    "_score" : 1.0,
    "_source" : {
      "name" : "Espresso Maker",
      "price" : 180,
      "in_stock" : 5,
      "tags" : [
        "kitchen appliances",
        "electrical",
        "coffee maker"
      ]
    }
  },
  {
    "_index" : "products",
    "_type" : "_doc",
    "_id" : "204",
    "_score" : 1.0,
    "_source" : {
      "name" : "Pressure Fryer",
      "price" : 120,
      "in_stock" : 50,
      "tags" : [
        "air fryer",
        "kitchen appliances",
        "electric fryer",
        "fryer",
        "health fryer"
      ]
    }
  }
]
  }
}

Upon querying the data using the query below I am only matching six records: Query - 1

GET /products/_search
{
  "query": {"terms" : {"tags": ["kitchen appliances","electricals"]}}
}

The document id's matched are (201,205,206,209,210,211)

When I executed the below query then I am matching 11 records: Query-2

GET /products/_search
{
  "query": {"terms" : {"tags.keyword": ["kitchen appliances","electricals"]}}
}

The document id's that matched for the second query are : (200,201,202,203,204,205,206,207,209,210,211)

Can someone explain what is the difference between the two queries and why Query-1 is a subset of Query-2 even though both queries are being executed on the same field ?

The queries are not executed on the same field. `tags` and `tags.keyword` are two different fields with two different types. — Val, Apr 19 '21 at 05:09

ESCoder · Accepted Answer · 2021-04-19T05:39:13.813

It is better to use the match query if you have a text type field.

term query doesn't perform any analysis on the term. It returns the documents that contain exact term matching documents.

terms query works on exact terms. It returns those documents that have 1 or more exact terms.

QUERY 1:

{
  "query": {
    "terms": {
      "tags": [
        "kitchen appliances",
        "electricals"
      ]
    }
  }
}

Search Result is

"hits": [
      {
        "_index": "67155973",
        "_type": "_doc",
        "_id": "3",
        "_score": 1.0,
        "_source": {
          "name": "Microwave Oven",
          "price": 100,
          "in_stock": 50,
          "tags": [
            "kitchen appliances",
            "electricals",
            "oven",
            "oven toaster",
            "microwave"
          ]
        }
      },
      {
        "_index": "67155973",
        "_type": "_doc",
        "_id": "4",
        "_score": 1.0,
        "_source": {
          "name": "Mixer Grinder",
          "price": 55,
          "in_stock": 130,
          "tags": [
            "kitchen appliances",
            "electricals",
            "mixer",
            "grinder",
            "juicer",
            "food processor"
          ]
        }
      },
      {
        "_index": "67155973",
        "_type": "_doc",
        "_id": "7",
        "_score": 1.0,
        "_source": {
          "name": "Rice Maker",
          "price": 85,
          "in_stock": 60,
          "tags": [
            "kitchen appliances",
            "electricals",
            "electric rice cooker",
            "electric pressure cooker"
          ]
        }
      },
      {
        "_index": "67155973",
        "_type": "_doc",
        "_id": "8",
        "_score": 1.0,
        "_source": {
          "name": "Induction Cooktop",
          "price": 30,
          "in_stock": 150,
          "tags": [
            "kitchen appliances",
            "electricals",
            "hot plate heater",
            "electric hot place",
            "induction cooker",
            "induction stove"
          ]
        }
      },
      {
        "_index": "67155973",
        "_type": "_doc",
        "_id": "9",
        "_score": 1.0,
        "_source": {
          "name": "Coffee Maker",
          "price": 50,
          "in_stock": 100,
          "tags": [
            "kitchen appliances",
            "electricals"
          ]
        }
      },
      {
        "_index": "67155973",
        "_type": "_doc",
        "_id": "14",
        "_score": 1.0,
        "_source": {
          "name": "Milk Frother",
          "price": 25,
          "in_stock": 15,
          "tags": [
            "kitchen appliances",
            "electricals",
            "milk"
          ]
        }
      }
    ]

As mentioned in the documentation

The term query does not analyze the search term. The term query only searches for the exact term you provide. This means the term query may return poor or no results when searching text fields.

QUERY 2:

{
  "query": {
    "terms": {
      "tags.keyword": [
        "kitchen appliances",
        "electricals"
      ]
    }
  }
}

In the above query, you are using tags.keyword field which uses the keyword analyzer instead of the standard analyzer. Here the query searches for the exact terms i.e "kitchen appliances" OR "electricals", and therefore returns 11 documents.

Yes Sir I did go through the answer and it mostly explained the working of the keyword analyzer and the standard analyzer but where my doubt arises is that in both the queries we used the "terms" query and as I understand terms query are not analyzed. Looking forward for an answer to this. — Sarvagya Dubey, Apr 21 '21 at 17:59
@SarvagyaDubey terms query does not analyze the text. It is recommended to use term/terms query with `keyword` type field to get accurate results. It is mentioned in the elasticsearch official documentation also that, **term query may return poor or no results when searching text fields.** --> https://www.elastic.co/guide/en/elasticsearch/reference/7.12/query-dsl-term-query.html#avoid-term-query-text-fields — ESCoder, Apr 21 '21 at 18:07

Why is there a difference in the search results when querying elasticsearch using a term query?

1 Answers1