Scope Elasticsearch Results to Specific Ids

Question

I have a question about the Elasticsearch DSL.

I would like to do a full text search, but scope the searchable records to a specific array of database ids.

In SQL world, it would be the functional equivalent of WHERE id IN(1, 2, 3, 4).

I've been researching, but I find the Elasticsearch query DSL documentation a little cryptic and devoid of useful examples. Can anyone point me in the right direction?

score 11 · Accepted Answer · answered Jan 22 '16 at 14:14

11

Here is an example query which might work for you. This assumes that the _all field is enabled on your index (which is the default). It will do a full text search across all the fields in your index. Additionally, with the added ids filter, the query will exclude any document whose id is not in the given array.

{
  "bool": {
    "must": {
      "match": {
        "_all": "your search text"
      }
    },
    "filter": {
      "ids": {
        "values": ["1","2","3","4"]
      }
    }
  }
}

Hope this helps!

answered Jan 22 '16 at 14:14

BrookeB

1,709
13
22

Thanks Brooke! That is exactly what I was looking for. The nuances of syntax are very very helpful. Much gratitude. – mindtonic Jan 22 '16 at 15:42
Yeah, it's pretty powerful. I'm definitely loving it, just need a little help navigating the syntax. Thanks again! – mindtonic Jan 22 '16 at 15:47
There is definitely a learning curve, coming from a SQL background. When you get a chance, I highly recommend going through the guide. It is invaluable, and much easier to follow than just the documentation. https://www.elastic.co/guide/en/elasticsearch/guide/current/index.html – BrookeB Jan 22 '16 at 16:40

score 3 · Answer 2 · answered Apr 21 '21 at 09:02

As discussed by Ali Beyad, ids field in the query can do that for you. Just to complement his answer, I am giving an working example. In case anyone in the future needs it.

GET index_name/_search
{
  "query": {
    "bool": {
      "must": [
        {
          "match": {
            "field": "your query"
          }
        },
        {
          "ids" : {
            "values" : ["0aRM6ngBFlDmSSLpu_J4", "0qRM6ngBFlDmSSLpu_J4"]
          }
        }
      ]
    }
  }
}

score 2 · Answer 3 · answered Jan 28 '21 at 23:55

2

According to es doc, you can

Returns documents based on their IDs.

GET /_search
{
  "query": {
    "ids" : {
      "values" : ["1", "4", "100"]
    }
  }
}

answered Jan 28 '21 at 23:55

Sean Gao

66
4

score 1 · Answer 4 · answered Jan 21 '16 at 02:29

1

You can create a bool query that contains an Ids query in a MUST clause: https://www.elastic.co/guide/en/elasticsearch/reference/2.0/query-dsl-ids-query.html

By using a MUST clause in a bool query, your search will be further limited by the Ids you specify. I'm assuming here by Ids you mean the _id value for your documents.

answered Jan 21 '16 at 02:29

Ali Beyad

11
1

Ali Beyad, thanks for the reply! Would you be so kind as to provide a full example of what the query json would look like? I'm wanting to do a full text search across all indexed fields and include the MUST clause, but everything I have tried results in syntax errors and I cannot find a good written example anywhere. – mindtonic Jan 21 '16 at 18:25
@mindtonic : so you want both the full-text search and the id search in the same request ? – gfd Jan 22 '16 at 10:23
I would also like to see a query example. Adding it to a condition within a bool should doesn't seem to do anything. – Tum Sep 06 '19 at 14:51

score 0 · Answer 5 · answered Feb 28 '21 at 19:51

0

With elasticaBundle symfony 5.2

    $query = new Query();
    $IdsQuery = new Query\Ids();
    $IdsQuery->setIds($id);
    $query->setQuery($IdsQuery);
    $this->finder->find($query, $limit);

answered Feb 28 '21 at 19:51

Moccine

1,021
6
14

score 0 · Answer 6 · answered Oct 11 '22 at 18:28

You have two options.

The ids query:

GET index/_search
{
  "query": {
    "ids": {
      "values": ["1, 2, 3"]
    }
  }
}

or

The terms query:

GET index/_search
{
  "query": {
    "terms": {
      "yourNonPrimaryIdField": ["1", "2","3"]
    }
  }
}

The ids query targets the document's internal _id field (= the primary ID). But it often happens that documents contain secondary (and more) IDs which you'd target thru the terms query.

Note that if your secondary IDs contain uppercase chars and you don't set their field's mapping to keyword, they'll be normalized (and lowercased) and the terms query will appear broken because it only works with exact matches. More on this here: Only getting results when elasticsearch is case sensitive

Scope Elasticsearch Results to Specific Ids

6 Answers6