Delete data older than 10 days in elasticsearch

Question

I am new to elasticsearch and I want to delete documents in my elasticsearch index which are older than 10 days. I want to keep only last 10 days of data.So is there any way to delete last 11nth day index automatically. What I have tried..

DELETE logstash-*/_query
{
 "query": {
   "range": {
     "@timestamp": {
       "lte": "now-10d"
      }
    }
  }
}

Error I'm getting while running on kibana dev tools

{
"error": "Incorrect HTTP method for uri [/logstash-*/_query?pretty] and method [DELETE], 
allowed: [POST]",
"status": 405
}

Please help to resolve this issues.

score 12 · Answer 1 · answered Sep 17 '19 at 13:14

12

You need to leverage the Delete by Query Endpoint, like this:

use POST           use this endpoint
 |                      |
 V                      V
POST logstash-*/_delete_by_query
{
 "query": {
   "range": {
     "@timestamp": {
       "lte": "now-10d"
      }
    }
  }
}
  ^
  |
the query part is fine !!

answered Sep 17 '19 at 13:14

Val

207,596
13
358
360

Thanks for your response but its not working.... { "took" : 1, "timed_out" : false, "total" : 0, "deleted" : 0, "batches" : 0, "version_conflicts" : 0, "noops" : 0, "retries" : { "bulk" : 0, "search" : 0 }, "throttled_millis" : 0, "requests_per_second" : -1.0, "throttled_until_millis" : 0, "failures" : [ ] } . I'm getting such output and its not reflecting any changes. – Kavya Sep 17 '19 at 17:09
Well with the response you showed that @Val 's answer fixed the original issue. Can you verify that there are even documents in that time range? – apt-get_install_skill Sep 17 '19 at 17:33
That means no documents matched the range. Replace `_delete_by_query` with `_search` and no documents should come up. – Val Sep 17 '19 at 18:08

score 1 · Answer 2 · answered Sep 18 '19 at 18:26

1

I am describe diffent approach, then @Val have suggested. You can create 10 indexes (index per day) and each day delete one of the indexes - oldest one.

Pros: it is very easy to delete or archive old data
Cons: you need to rewrite your queries if you need to search all days data.

answered Sep 18 '19 at 18:26

ceth

44,198
62
180
289

You can do same thing using ILM policy instead of through API each time. – Rahul Dec 11 '20 at 10:26

Delete data older than 10 days in elasticsearch

2 Answers2

Linked