1

I am using ES 2.3.3 and Logstash 2.3.3. I have been using Logstash to send data and map them into ES for indexing, i.e.logstash-{Date}. And I would only like to keep the file which is the latest 1 year. Any index over a year should be deleted. I was using 3.5.1 before. The way for me to delete the indexes is to input a command everyday.

curator --host 10.0.0.2 delete indices --older-than 30 --time-unit days \
   --timestring '%Y.%m.%d'

Recently, I then I have upgraded curator 3.5.1 to curator 4. However I could not find where curator is stored even though I have read through the examples from https://www.elastic.co/guide/en/elasticsearch/client/curator/current/command-line.html Therefore, I would wanna know where will the configuration file be and why there will be a missing of action_file? Does that mean I need to create a new .curator directory and also my own curator.yml and action.yml file?

And after I have created my action.yml file, should I just follow the https://www.elastic.co/guide/en/elasticsearch/client/curator/current/examples.html#ex_delete_indices and add this part into my action.yml file in order to delete logstash indexes over a year?

Thanks

John Y
  • 14,123
  • 2
  • 48
  • 72
Kennedy Kan
  • 273
  • 1
  • 7
  • 20
  • Can you please give more information about what it is you're trying to accomplish? What do your indices look like? Are they time series? Do they have regular naming patterns? What operating environment (OS, Elasticsearch version, etc.) are you working under? – untergeek Jul 04 '16 at 13:24
  • Yes, I have updated my question and I hope it can give a better picture of my needs. Thanks. – Kennedy Kan Jul 05 '16 at 02:21
  • You can take the reference from this link http://stackoverflow.com/a/42268400/2874585 – Sachchit Bansal Feb 16 '17 at 08:25

2 Answers2

6

The configuration file can be anywhere, so long as you launch Curator with the --config flag:

curator --config /path/to/curator_config.yml

However, if you make a .curator path in the home directory of the user who will be running Curator (via cron, ostensibly), it will look there for a file called curator.yml, e.g. /home/username/.curator/curator.yml

With that file properly configured in that location, Curator will not require the --config flag.

Curator just uses the final argument as the action file:

» curator --help
Usage: curator [OPTIONS] ACTION_FILE

  Curator for Elasticsearch indices.

  See http://elastic.co/guide/en/elasticsearch/client/curator/current

Options:
  --config PATH  Path to configuration file. Default: ~/.curator/curator.yml
  --dry-run      Do not perform any changes.
  --version      Show the version and exit.
  --help         Show this message and exit.

An example of running Curator with the default configuration file in $HOME/.curator/curator.yml would be:

curator /path/to/actionfile.yml

And with a custom configuration file:

curator --config /path/to/curator_config.yml /path/to/actionfile.yml

Following the action file examples is a great place to start. Feel free to experiment with new configurations, but be sure to use the --dry-run flag when doing so, to prevent any action from being taken while testing.

untergeek
  • 863
  • 4
  • 13
1

Curator also comes with curator_cli where you can run below command for a quick go.

curator_cli --host https://full-url:port delete_indices --ignore_empty_list --filter_list '[{"filtertype":"pattern","kind":"prefix","value":"logstash-"}]'
kartik
  • 2,097
  • 3
  • 21
  • 31
  • While this is true, the `curator_cli` feature did not exist for Curator version 4 at the time the question was asked. – untergeek Mar 07 '18 at 13:38