0

I am very new in using Logstash. I have two kinds of log,

Pattern 1 : --2019-05-09 08:53:45.057 -INFO 11736 --- [ntainer#1-0-C-1] c.s.s.service.MessageLogServiceImpl : [adc7fd862db5307a688817198046b284dbb12b9347bed9067320caa49d8efa381557392024151] Event => Message Status Change [Start Time : 09052019 08:53:44] : CUSTOM_PROCESSING_COMPLETED

Pattern 2 : --2019-05-09 06:49:05.590 -TRACE 6293 --- [ntainer#0-0-C-1] c.s.s.service.MessageLogServiceImpl : [41a6811cbc1c66eda0e942712a12a003d6bf4654b3edb6d24bf159b592afc64f1557384545548] Event => Message Failure Identified : INVALID_STRUCTURE

Though there are many more other lines, but I want to consider only these two types. Hence I used below filter,

 grok {
     #Event : message status change
     match => {
         "message" => "--(?<logtime>[^\]]*) -%{LOGLEVEL:level} (?<pid>\d+) --- \[(?<thread>[^\]]+)] (?<classname>[\w.]+)\s+: \[(?<token>[^\]]+)] Event \=> Message Status Change \[Start Time : (?<start>[^\]]*)\] : (?<status>[\w]+)"
     }
     add_field => {
         "event" => "message_status_change"
     }
 }


 grok {
     #Event : message failure
     match => {
         "message" => "--(?<logtime>[^\]]*) -%{LOGLEVEL:level} (?<pid>\d+) --- \[(?<thread>[^\]]+)] (?<classname>[\w.]+)\s+: \[(?<token>[^\]]+)] Event \=> Message Failure Identified : (?<code>[\w]+)"
     }
     add_field => {
         "event" => "message_failure"
     }
 }

I have also noticed that both of these grok patterns work individually (if I comment one, then other one works perfectly). Logstash server also ok when both patterns are active. But it raises a grokparse error when both of them is open and a new line is added in the log file.

Also I want to know, though I am configured the input to read from a file from beginning, it is not reading even after server restart unless I add a new line in the log. Why this behaviour?

Thanks in advance.

Abhishek Chatterjee
  • 1,962
  • 2
  • 23
  • 31
  • For the second part of the question, see [this](https://stackoverflow.com/questions/19546900/how-to-force-logstash-to-reparse-a-file). Logstash remembers what parts of the file have already been read and so won't re-read lines which has already been read. – baudsp May 21 '19 at 15:08
  • For the first part of the question, you can put the two pattern in the same grok filter. Or use `if [message] ~= /Message Status Change/ { grok with pattern 1} else { grok with pattern 2}` – baudsp May 21 '19 at 15:10

0 Answers0