2

I am getting logs from logstash and sending messages which contains ERROR to riemann server.

logsash message : message" => "Jul 10 04:34:47 : [ERROR] [host] Sample abc123"

In riemann I need to write a logic to find if the same ERROR occurs more than 2 times for the same sample(i.e sample abc 123) in 5min then I have to alert /print on the console(currently I am trying to print to test)

below is my logic to achieve this, currently it prints only nil and not the message, can some one please help me here.

(by [:message]      

 ;; over time windows of 5 seconds...
    (fixed-time-window 5

        ;; calculate the average value of the metric and emit an average (summary) event
        (combine folds/mean

            ;; collect the summary event over the last 3 fixed-time-windows
            (moving-event-window 2

                ;;find the summary event with the minimum average metric
                (combine folds/minimum
                     (where (> metric 2.0)
                       prn
                       ;;(email "xyz")
                     )
                )
            )
        )
    )
)

(It prints message every 2sec, if I use just fixed-time-window and nothing else but I want to print only if same sample occurs 2times in 5min)

Riemann event for example: 

:service "logstash", 
:time 1468920656, 
:state "critical", 
:host "0.0.0.0", 
:ttl nil, 
:message "Jul 11 04:34:47  <hostname>: [ERROR] [xyz] - Sample abc123 causes problem for ...""
Vinod HC
  • 1,557
  • 5
  • 20
  • 38
  • Could you include the output of printing one of the events as it arrives (suitably redacted) so I can see what an event l looks like. Are there a fixed number of total different messages sent? I ask because `by` can cause problems if you split the steam over an large set of possible branches and want to know if it's safe to use in an answer. Also what is the throughput of these events, more or less than 1k/second? – Arthur Ulfeldt Jul 20 '16 at 16:18
  • Thanks Arthur, I have added riemann event.. currently I am sending similar events to riemann server every 2sec.. Do I need to send more to test this? – Vinod HC Jul 20 '16 at 17:40
  • No, I just wanted to get a sense of scale. event rates among riemann users range from one/hour to 20,000/second per processor and using `by` in the latter case can use substantial ram. – Arthur Ulfeldt Jul 20 '16 at 21:03

0 Answers0