Reading from rotating log files in logstash

Question

As per the documentation of logstash's file plugin, the section on File Rotation says the following:

To support programs that write to the rotated file for some time after the rotation has taken place, include both the original filename and the rotated filename (e.g. /var/log/syslog and /var/log/syslog.1) in the filename patterns to watch (the path option).

If anyone can clarify how to specify two filenames in the path configuration, that will be of great help as I did not find an exact example. Some examples suggest to use wild-cards like /var/log/syslog*, however I am looking for an example that achieves exactly what is said in documentation - two filenames in the path option.

score 3 · Accepted Answer · answered Aug 04 '16 at 10:04

The attribute path is an array and thus you can specify multiple files as follows:

input {
   file{
       path => [ "/var/log/syslog.log", "/var/log/syslog1.log"]
   }
}

You can also use * notation for name or directory as follows:

input {
   file{
       path => [ "/var/log/syslog.log", "/var/log/syslog1.log", "/var/log/*.log", "/var/*/*.log"]
   }
}

When you specify path as /var/*/*.log it does a recursive search to get all files with .log extension.

Reference Documentation

Reading from rotating log files in logstash

1 Answers1