2

According to this issue, elasticsearch supports using percolate with index (a single document) or bulk (multiple documents). No example is given for bulk, so I'm going by the issue's title that this functionality was added 5 years ago.

Unfortunately, I can't find any information about this functionality being available in the ruby API, elasticsearch-ruby.

Does anyone know if it's available, or perhaps have a code sample?

Thanks.

Update:

This page describes how to percolate while indexing a single document, and claims that it's possible in bulk. Now, how to do that in elasticsearch-ruby?

Looks like the NEST library can do it (see the bottom of the page), if I cared to rewrite my project in .Net.

Alain Collins
  • 16,268
  • 2
  • 32
  • 55
  • may be this will help a bit. http://stackoverflow.com/questions/29427121/how-to-know-if-a-geo-coordinate-lies-within-a-geo-polygon-in-elasticsearch – Manjit Kumar Jul 03 '15 at 07:41
  • @ManjitKumar, thanks, but they aren't even indexing documents, let alone doing it in bulk... – Alain Collins Jul 04 '15 at 09:05
  • as Andrei stated, you can not index and percolate documents in one step since 1.0. The link to the document is from 2011, there have been so many changes since. The NEST library may offer you a wrapper which probably uses the bulk result, then percolate indexed documents. – Julien C. Jul 06 '15 at 13:34
  • Sad that features just disappear. – Alain Collins Jul 06 '15 at 16:14

2 Answers2

2

There shouldn't be any special approach in Ruby, other than a simple _bulk operation specifying the _index as your index and as _type the .percolator:

POST /_bulk
{"index":{"_index":"some_index","_type":".percolator","_id":"1"}}
{"query":{"match":{"whatever_field":"some value 1"}}}
{"index":{"_index":"some_index","_type":".percolator","_id":"2"}}
{"query":{"match":{"whatever_field":"some value 2"}}}
{"index":{"_index":"some_index","_type":".percolator","_id":"3"}}
{"query":{"match":{"whatever_field":"some value 3"}}}

For sending multiple percolate requests, there is mpercolate which, initially, was created as a result of percolate in bulk feature request and ended up being called multi percolate api. And I see that elasticsearch-ruby has support for it.

The ?percolate=* feature for bulk it seems is not in the list of arguments for _bulk and, in fact, there is an issue opened for this: github.com/elastic/elasticsearch-ruby/issues/176.

LATER EDIT: I looked at this again and I'm more inclined to believe this feature has been removed completely in ES 1.0.0, following the percolator redesign github issue. I'm not seeing a specific statement about this being removed, but the source code related to bulk indexing with percolator option following that redesign has been deleted. Also, the documentation doesn't specify this option anywhere. Usually, when this happens is not an oversight but the functionality is simply not there. Still related to the documentation - don't forget that any resources you found online about this are at least two years old and I would take them with a grain of salt before assuming they are still valid.

I wouldn't be surprised if you test this in Elasticsearch (no ruby, nest or whatever client) with curl or Sense and see that it doesn't work. I tested this in 1.6 just now and I'm not seeing the functionality.

Andrei Stefan
  • 51,654
  • 6
  • 98
  • 89
  • I thought the .percolator type was where percolate queries were stored. I'm not trying to *create* percolate queries in bulk, but to *index* documents in bulk and have the percolate results returned to me in the response (as they can be when you index a single document). If I missed something in your answer, please elaborate. – Alain Collins Jul 02 '15 at 21:56
  • Oh, the other way around. Updated my answer. – Andrei Stefan Jul 02 '15 at 21:59
  • mpercolate is for sending documents (or the ids of existing documents) to be percolated, not for indexing and percolating in one step. – Alain Collins Jul 02 '15 at 22:07
  • Then I don't understand what you are looking for. – Andrei Stefan Jul 02 '15 at 22:08
  • 1
    "localhost:9200/twitter/tweet/1?percolate=*" will index and percolate one document in the same step, via curl. I want to index and percolate *bulk* documents in *ruby*. (And thanks for your attention!) – Alain Collins Jul 02 '15 at 22:11
  • It seems it's not in the [list of arguments for `_bulk`](https://github.com/elastic/elasticsearch-ruby/blob/master/elasticsearch-api/lib/elasticsearch/api/actions/bulk.rb#L47) and, in fact, there is an issue opened for this: https://github.com/elastic/elasticsearch-ruby/issues/176. – Andrei Stefan Jul 02 '15 at 22:39
  • The whole point of this SO question was to get a modern definitive answer. Unfortunately, we'll have to settle for "oh, looks like it was removed for some reason" and send the documents to elasticsearch twice for every operation. Progress! – Alain Collins Jul 06 '15 at 16:15
  • :-) You can, also, create a "feature" github issue for this and see what feedback you get. – Andrei Stefan Jul 07 '15 at 00:04
0

The latest master documentation (of currently unreleased Elasticsearch 6.x) has a section on percolating multiple documents. This is new functionality scheduled for release in 6.X, but not in the 6.0 release candidates.

Izak Marais
  • 131
  • 4