Is it possible to put the results of an ElasticSearch aggregation back into the index?

Question

So I have an index with ping data from host A to host B and my data looks like this:

{
  "@version" => "1",
  "@timestamp" => "2014-07-17T21:17:34.030Z",
  "host" => "host_a",
  "to_host" => "host_b",
  "value" => "25.6",
  "from_host" => "host_a",
  "stat_type" => "ping"
}

The goal is to also store 90th percentile data for the value of the ping ("value" in the above) on a moving window e.g. last hour, last day etc.

I know I can do this with an aggregation but my question is as follows:

Does ElasticSearch support a way to automatically add the output of an aggregation (or query for that matter) back into the index?

I know I could probably just take the output, tweak the fields and then put the data back in using some helper application but was curious as to if this is possible only using ES.

To use an equivalent SQL example, I would be looking for something like this:

create table agg
select id, count(*) as counts
from data
group by id;

Answer 1

Perhaps a bit off of what you are looking for but you could do this using Logstash, which is a part of Elasticsearch http://www.elasticsearch.com/blog/welcome-jordan-logstash/ .

I won't go into a lot of detail here (there are many Logstash tutorials out there) but appending the results of your aggregation query to a logfile and having logstash automatically take the results and load it into an Elasticsearch index is pretty trivial. This tutorial covers all you need to get logstash reading from a log file and automatically indexing the contents:

http://logstash.net/docs/1.4.2/tutorials/getting-started-with-logstash

you could of course do this in code but most of the work here is taken care of, once you have logstash setup all you need to do is append your results to the log file.