How to nest bool queries?

Question

I am building a search query which dynamically adds a set of constraints (bool) to the query. The general expected structure is as follows

OR (

AND (
    condition
    condition
    ...
)

AND (
    condition
    condition
    ...
)

)

In other words I have a set (one or more) of conditions which must all be met (AND above). There may be several of such sets, any of them should be enough for the final match (the OR above).

An example of such structure, as generated by my code (this is the full API query, the generated part is "bool"):

{
  "query": {
    "bool": {
      "should": [
        {
          "bool": {
            "must": [
              {
                "term": {
                  "attack_ip": "10.89.7.117"
                }
              },
              {
                "term": {
                  "sentinel_port": "17"
                }
              }
            ]
          }
        },
        {
          "bool": {
            "must": [
              {
                "term": {
                  "attack_ip": "10.89.7.118"
                }
              }
            ]
          }
        }
      ]
    },
    "range": {
      "eventtime": {
        "gte": "2018-03-05T12:47:22.397+01:00"
      }
    }
  },
  "size": 0,
  "aggs": {
    "src": {
      "terms": {
        "field": "attack_ip",
        "size": 1000
      },
      "aggs": {
        "dst": {
          "terms": {
            "field": "sentinel_hostname_lan",
            "size": 2000
          }
        }
      }
    }
  }
}

My understanding of this query was:

if "attack_ip === 10.89.7.117" and "sentinel_port === 17"
or
if "attack_ip === 10.89.7.118"

the entry will match

Unfortunately I get upon calling Elasticsearch the error

"error": {
    "root_cause": [
      {
        "type": "parsing_exception",
        "reason": "[bool] malformed query, expected [END_OBJECT] but found [FIELD_NAME]",
        "line": 1,
        "col": 177
      }
    ],
    "type": "parsing_exception",
    "reason": "[bool] malformed query, expected [END_OBJECT] but found [FIELD_NAME]",
    "line": 1,
    "col": 177
  },
  "status": 400
}

What does this error mean?

EDIT

Following Piotr's answer, I tried to move the range constraint into the boolean part. I get the same error, though.

My query is available online for easier reading and reproduced below:

{
  "query": {
    "bool": {
      "must": [
        {
          "bool": {
            "should": [
              {
                "bool": {
                  "must": [
                    {
                      "term": {
                        "attack_ip": "10.89.7.117"
                      }
                    },
                    {
                      "term": {
                        "sentinel_port": "17"
                      }
                    }
                  ]
                }
              },
              {
                "bool": {
                  "must": [
                    {
                      "term": {
                        "attack_ip": "10.89.7.118"
                      }
                    }
                  ]
                }
              }
            ]
          }
        },
        {
          "range": {
            "eventtime": {
              "gte": "2018-03-05T13:55:27.927+01:00"
            }
          }
        }
      ]
    },
    "size": 0,
    "aggs": {
      "src": {
        "terms": {
          "field": "attack_ip",
          "size": 1000
        },
        "aggs": {
          "dst": {
            "terms": {
              "field": "sentinel_hostname_lan",
              "size": 2000
            }
          }
        }
      }
    }
  }
}

Possible duplicate with answer: https://stackoverflow.com/a/40755927/4614788 — Glenn Van Schil, Mar 05 '18 at 13:55
@GlennVanSchil: This is a good question about how ot use the bool in general - I directly used the documentation at https://www.elastic.co/guide/en/elasticsearch/guide/current/combining-filters.html. I do not see how it solves my current problem, though. — WoJ, Mar 05 '18 at 14:01

score 2 · Accepted Answer · edited Mar 05 '18 at 21:16

I think the problem you have is with range part. Try to move it inside the bool:

{
  "query": {
    "bool": {
      "should": [{
          "bool": {
            "must": [{
                "term": {
                  "attack_ip": "10.89.7.117"
                }
              },
              {
                "term": {
                  "sentinel_port": "17"
                }
              }
            ]
          }
        },
        {
          "term": {
            "attack_ip": "10.89.7.118"
          }
        }
      ],
      "must": {
        "range": {
          "eventtime": {
            "gte": "2018-03-05T12:47:22.397+01:00"
          }
        }
      }
    }
  },
  "size": 0,
  "aggs": {
    "src": {
      "terms": {
        "field": "attack_ip",
        "size": 1000
      },
      "aggs": {
        "dst": {
          "terms": {
            "field": "sentinel_hostname_lan",
            "size": 2000
          }
        }
      }
    }
  }
}

or move it to filter section:

{
  "query": {
    "bool": {
      "should": [{
          "bool": {
            "must": [{
                "term": {
                  "attack_ip": "10.89.7.117"
                }
              },
              {
                "term": {
                  "sentinel_port": "17"
                }
              }
            ]
          }
        },
        {
          "term": {
            "attack_ip": "10.89.7.118"
          }
        }
      ],
      "filter": {
        "bool": {
          "must": [{
            "range": {
              "eventtime": {
                "gte": "2018-03-05T12:47:22.397+01:00"
              }
            }
          }]
        }
      }
    }
  },
  "size": 0,
  "aggs": {
    "src": {
      "terms": {
        "field": "attack_ip",
        "size": 1000
      },
      "aggs": {
        "dst": {
          "terms": {
            "field": "sentinel_hostname_lan",
            "size": 2000
          }
        }
      }
    }
  }
}

I hope I formatted this correctly. Please let me know if you have any issues.

In the end, it is possible that you will need to specify minimum_should_match param for bool query to get correct results.

Thanks. I did not realize that the range should be part of the boolean section. I updated my question with the query (and a link to an online viewer which makes life easier) but I still have the same issue — WoJ, Mar 05 '18 at 14:03
I am analyzing your query and I do not understand one thing: why didn't you put two must conditions (the range and the set of ORs (should)) . that would translate to "the date must be in the range AND one or more of the blocks of terms must be matched") — WoJ, Mar 05 '18 at 14:17
You missed one thing in my code - you have to have "size" and "aggs" at the same level as "query", not inside "query". Take a look at: https://jsoneditoronline.org/?id=7923d877cf3c1dfc07eaf943715ed9a1 — Piotr Pradzynski, Mar 05 '18 at 14:57
Ah! Thank you! I completely missed the fact that I included the aggregation in the query. Now it works correctly. Thanks again. As for the size: 0, I am not interested in individual hits but only in the aggregations. — WoJ, Mar 05 '18 at 15:29

How to nest bool queries?

EDIT

1 Answers1