Elasticsearch - Want to sort by field in all indices where that particular field available or not if not then avoid it

Question

Currently, Getting result based on scoring but what i want to do is i want a result based on scoring + Field Status with value true/false.

If value is true then needed that results in priority but there is possibility that status field is not exist in all indices.

           "query" => [
                  'bool' => [
                     'filter' => $filter,
                     'must' => [
                     "multi_match" => [
                        'query' => "$string",
                        "type" => "cross_fields",
                        'fields' => ['field1','field2','field3'],
                        "minimum_should_match" => "80%"
                         ]
                    ]
                  ]
            ],
            "sort" => [
                    "_score",
                    [ "status" => ["order" => "desc","unmapped_type" => "boolean"] ]
            ],

But getting error below :

[type] => illegal_argument_exception
[reason] => Text fields are not optimised for operations that require per-document field data like aggregations and sorting, so these operations are disabled by default. Please use a keyword field instead. Alternatively, set fielddata=true on [status] in order to load field data by uninverting the inverted index. Note that this can use significant memory.

Anyone help me out to ignore for indices where that field not available or any other solution with this problem?

see this https://magento.stackexchange.com/a/332880/2867 – Mike Mar 16 '21 at 16:08 — Mike, Mar 16 '21 at 16:08

Amit · Accepted Answer · 2020-05-27T14:13:00.033

15

As discussed in the chat, the issue happened due to @jilesh

forget to delete the old index mapping and only upate the data that's what this thing was occurring.

Below answer is relevant when you get below error with proper setup

Text fields are not optimised for operations that require per-document field data like aggregations and sorting, so these operations are disabled by default. Please use a keyword field instead. Alternatively, set fielddata=true on [status] in order to load field data by uninverting the inverted index. Note that this can use significant memory.

In that case, please enable the field data on the field if you want to get rid of the error but beware it can cause performance issues.

Read more about the field data on official site.

You can enable it in your order field in your mapping as shown.

{
  "properties": {
    "order": { 
      "type":     "text",
      "fielddata": true
    }
  }
}

edited May 27 '20 at 14:13

answered May 27 '20 at 12:36

Amit

30,756
6
57
88

I am trying to sort by status field with descending order but main issue is status field not available in every indexes and issue causing with only indexes where this field not available. If something wrong with my query then please help me out to correct – jilesh May 27 '20 at 12:41
@jilesh, then you should use `exist` query with `must` clause to search only on those documents where this `order` field present refer https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl-exists-query.html#find-docs-null-values for more details – Amit May 27 '20 at 12:44
@jilesh, but the error which you mentioned is different, can you provide your index mapping as well? – Amit May 27 '20 at 12:45
```order``` is not a field field name is ```status``` i am trying to set status field in descending order. – jilesh May 27 '20 at 12:45
Not customize mapping from my end it's generated by default and i think it's different for every indices as per the fields available into it. – jilesh May 27 '20 at 12:47
@jilesh, oh yes, my bad , yeah as its generated default it would be text and that's the reason of error, can get the generated mapping using https://www.elastic.co/guide/en/elasticsearch/reference/current/indices-get-mapping.html and provide(formatted) in the question as it might be big – Amit May 27 '20 at 12:50
Let us [continue this discussion in chat](https://chat.stackoverflow.com/rooms/214740/discussion-between-jilesh-and-opster-elasticsearch-ninja). – jilesh May 27 '20 at 12:51

score 0 · Answer 2 · answered Mar 28 '22 at 23:43

0

works for me

curl -X PUT -H "Content-Type: application/json" \
    -d '{"properties": {"format": { "type":"text","fielddata": true}}}' \
    <your_host>:9200/<your_index>/_mapping

answered Mar 28 '22 at 23:43

Adán Escobar

1,729
9
15

score 0 · Answer 3 · answered Apr 20 '22 at 14:07

In my case, I had to use the aggregatable {fieldname}.keyword field. Here's an example using Nest .NET.

.Sort(s => s
    .Ascending(f => f.Field1.Suffix("keyword"))
    .Ascending(f => f.Field2.Suffix("keyword"))
    .Ascending(f => f.Field3.Suffix("keyword")))

Elasticsearch - Want to sort by field in all indices where that particular field available or not if not then avoid it

3 Answers3