1

I'm learning logstash and I'm using Kibana to see the logs. I would like to know if is there anyway to add fields using data from message property.

For example, the log is like this:

@timestamp:December 21st 2016, 21:39:12.444 port:47,144  
appid:%{[path]} host:172.18.0.5 levell:level message:
{"@timestamp":"2016-12-22T00:39:12.438+00:00","@version":1,"message":"Hello","logger_name":"com.empresa.miAlquiler.controllers.UserController","thread_name":"http-nio-7777-exec-1","level":"INFO","level_value":20000,
"HOSTNAME":"6f92ae402cb4","X-Span-Export":"false","X-B3-SpanId":"8f548829e9d18a8a","X-B3-TraceId":"8f548829e9d18a8a"}

My logstash conf is like:

filter {
grok {
match => {
  "message" =>
  "^%{TIMESTAMP_ISO8601:timestamp}\s+%{LOGLEVEL:level}\s+%{NUMBER:pid}\s+---\s+\[\s*%{USERNAME:thread}\s*\]\s+%{JAVAFILE:class}\s*:\s*%{DATA:themessage}(?:\n+(?<stacktrace>(?:.|\r|\n)+))?$"
}
}
 date {
match => [ "timestamp" , "yyyy-MM-dd HH:mm:ss.SSS" ]
}
mutate {
remove_field => ["@version"]
add_field => {
  "appid" => "%{[path]}"
}
add_field => {
  "levell" => "level"
}

} }

I would like to take level (in the log is INFO), and message (in the log is Hello) and add them as fields.

Is there anyway to do that?

Kamil Naja
  • 6,267
  • 6
  • 33
  • 47
AleGallagher
  • 1,745
  • 7
  • 30
  • 40
  • I could answer the question with this post: http://stackoverflow.com/questions/33937936/how-to-parse-json-in-logstash-grok-from-a-text-file-line – AleGallagher Jan 03 '17 at 00:44

1 Answers1

3

What if you do something like this using mutate:

filter { 
  mutate { 
    add_field => ["newfield", "%{appid} %{levell}"] <-- this should concat both your appid and level to a new field
  } 
}

You might have a look at this thread.

Kulasangar
  • 9,046
  • 5
  • 51
  • 82
  • Thank you for your answer @Kulasangar ! , I traid to do `add_field => {"tipo" => "%{evel}"}` , but the field shows explicitly: "%{level}", insted of the value, for example "INFO". – AleGallagher Dec 27 '16 at 00:41
  • Are you sure your mutate is working? I mean where you're getting the `level value: add_field => { "levell" => "level" } – Kulasangar Dec 27 '16 at 04:10
  • I think so, the logstash conf file, looks like: `mutate { remove_field => ["@version"] add_field => {"level_type" => "%{levell}"} }` I don't know what to do. – AleGallagher Dec 28 '16 at 00:51
  • could be the problem that The property appears in the log, but whitin "message" field ? – AleGallagher Dec 29 '16 at 03:01