Filtering empty or non empty array in Elasticsearch Kibana

Question

How can I search for an empty array or non empty array field in Kibana? To be more precise: There is an api that is used for queries and these requests/responses are logged. The search may result in an empty array if no elements have been found: 'response: []'. In other cases this "Response" field is an array populated with objects: 'response: [{"myProp": "something"}, {"myProp": "something2"}]'.
I've tried to use DSL queries, nested searches and other Stackoverflow answers but without success. For the following I get syntax error (or for any other solution I try to use) in the Kibana DSL query:

"must_not": {
  "script": {
    "script": "response.size() > 0"
  }
}

or using the following results in an Internal Server Error:

{
  "script": "response.size > 0"
}

The best thing to do is to add another integer field containing the number of results you have in the array. Then the query simply becomes a range query on that field checking for == 0 or > 0. Much more performant than messing with scripts. — Val, Mar 25 '21 at 14:54

score 1 · Answer 1 · answered Mar 25 '21 at 14:47

If your response field is NOT defined as nested, you'll need to target one of the array objects' keys instead of the array itself. You should preferably pick a key that's present in all of the child objects -- in your case myProp.

Now, if myProp has a defined keyword sub-mapping, you could do:

{
  "script": {
    "script": {
      "source": "doc['response.myProp'].size() > 0",
      "lang": "painless"
    }
  }
}

which basically says, look at the length of the doc values array and infer the parent array length based on that.

If your response is indeed nested, it gets even tricker. I've explained the reasons in my answers to:

but the principle stays the same — you'd extract the contents of the nested array objects onto a flattened level where it's easier to compute the resulting array length.

Now I have better understanding of these kind of objects. Right now creating indexes, is out of scope for me; I could not filter the data that works as intended. On the bright side I had no syntax error nor server error, so this gave me the right direction. — Daniel, Mar 30 '21 at 10:20

Filtering empty or non empty array in Elasticsearch Kibana

1 Answers1

Linked