0

I have an issue where logstash in not picking up the log file and pushing thing logs to Kibana. I have to restart the logstash process each time before it is picked up.

See example of logstash conf file

input {
  # one entry per topic file
  file {
    path => "/tmp/log-completed-backups.log"
    start_position => beginning
    type => "checkingfiles"
  }
}

filter {
  if "checkingfiles" in [type] {
    csv {
      separator => " "
      columns => ['cdate', 'ctime', 'bmonth', 'bday']
      add_tag => ["idam_filtered"]
    }
    mutate {
      rename => { "cdate" => "cdate" }
      rename => { "ctime" => "ctime" }
      rename => { "bmonth" => "bmonth" }
      rename => { "bday" => "bday" }

      add_field => {
        "namespace" => "${LB_SITE_NAME}"
      }
    }
  }
}
MrSimple
  • 599
  • 4
  • 14
Mikey R
  • 103
  • 2
  • 14
  • You got an elasticearch output in your configuration, right? Also note that logstash remember which lines it has read in a file and won't read them again (see https://stackoverflow.com/questions/19546900/how-to-force-logstash-to-reparse-a-file). – baudsp Feb 09 '18 at 10:33
  • @baudsp - Yes I have an output configuration file. – Mikey R Feb 09 '18 at 13:18
  • Okay. So I don't have any other idea. – baudsp Feb 09 '18 at 13:40
  • Would the formatting have anything to do with why the log file isn't being picked up automatically? – Mikey R Feb 12 '18 at 13:35
  • I don't understand. If there was a problem in the logstash configuration, logstash would not start, with an error message in the log like `Cannot create pipeline {:reason=>"Expected one of ...`. If it's a problem in the csv file, I don't know what its behavior would be. – baudsp Feb 12 '18 at 14:06

0 Answers0