3

I am running a java application in GKE and monitoring logs in Log explorer. Java application is writing logs to stdout and as far as I understand GKE agent parse it and send it to log explorer. What I found is that the log explorer shows WARN and ERROR messages with severity INFO. I figured out that I can't change the default parser of logs and configured logback to represent java logs in JSON format suitable for GCP (I used implementation from this answer), here is an example:

{"message":"2022-02-17 12:42:05.000 [QuartzScheduler_Worker-8] DEBUG some debug message","timestamp":{"seconds":1645101725,"nanos":0},"thread":"QuartzScheduler_Worker-8","severity":"DEBUG"}
{"message":"2022-02-17 12:42:05.008 [QuartzScheduler_Worker-8] INFO some info message","timestamp":{"seconds":1645101725,"nanos":8000000},"thread":"QuartzScheduler_Worker-8","severity":"INFO"}
{"message":"2022-02-17 12:42:05.009 [QuartzScheduler_Worker-8] ERROR some error message","timestamp":{"seconds":1645101725,"nanos":9000000},"thread":"QuartzScheduler_Worker-8","severity":"ERROR"}

But it didn't help at all.

Please point me out where I am wrong with JSON format or maybe I need to configure something additionally on the GCP side. I've checked the official doc regarding log JSON format and I don't understand what I am missing.

Kirill Liubun
  • 1,965
  • 1
  • 17
  • 35

1 Answers1

3

According to the documentation link 1 & link 2

Severities: By default, logs written to the standard output are on the INFO level and logs written to the standard error are on the ERROR level. Structured logs can include a severity field, which defines the log's severity.

If you're using Google Kubernetes Engine or the App Engine flexible environment, you can write structured logs as JSON objects serialized on a single line to stdout or stderr. The Logging agent then sends the structured logs to Cloud Logging as the jsonPayload of the LogEntry structure.

If the manual implementation is not working, you may try to:

  • Directly send logs to Cloud Logging API

  • Use this official Java logback lib (note: it's currently a WiP)

Atef Hares
  • 4,715
  • 3
  • 29
  • 61