2

First let me explain the setup/system:

I have a virtual Ubuntu 18.04 machine running via GCP Compute Engine. I have followed the GCP documentation and have installed the structured logging agent via apt-get.

On this machine I have a golang webservice, running inside a Docker container, which uses Logrus to put out logs that look like this:

{"message":"some interesting thing occured","severity":"INFO","timestamp":{"seconds":1599668988,"nanos":567943787}}

In order to get these logs from the machine / container to GCP Logs I have created a new fluentd configuration file /etc/google-fluentd/config.d/docker.conf, as described in fluentd documentation:

<source>
  @type forward
  port 24224
  bind 0.0.0.0
</source>

<filter docker.**>
  @type parser
  format json
  key_name log
  reserve_data true
</filter>

As well as the Docker daemon to send logs for this container to fluentd, as described in Docker documentation: docker run --log-driver=fluentd ...

The limitation I am facing is that the logs do arrive in GCP Logs Viewer, but the fields of my service are not correctly parsed, and therefore I am not able to sort by severity and so on. The logs that arrive in GCP look like this:

{
  "insertId": "xxxxxxxxxxxxxxxx",
  "jsonPayload": {
    "stream": "stderr",
    "log": "{\"message\":\"some interesting thing occured\",\"severity\":\"INFO\",\"timestamp\":{\"seconds\":1599668988,\"nanos\":567943787}}\n"
  },
  "resource": {
    "type": "gce_instance",
    "labels": {
      "instance_id": "22222222222222222",
      "project_id": "xxxxxxx-333333",
      "zone": "us-east1-b"
    }
  },
  "timestamp": "2020-09-09T16:29:48.568042072Z",
  "labels": {
    "compute.googleapis.com/resource_name": "xxxxxxxx"
  },
  "logName": "some/folder/logs/docker.var.lib.docker.containers.7ec67d861ea6d2dbf20db456dad4e7efc43b91a498580dda325f76170b502509.7ec67d861ea6d2dbf20db456dad4e7efc43b91a498580dda325f76170b502509-json.log",
  "receiveTimestamp": "2020-09-09T16:33:29.159377657Z"
}

What additional steps do I need to take so that the logs are understood by GCP Logs Viewer? I though that the filter/parser configuration for fluentd is enough to achieve this, in particular the section about.

  format json
  key_name log
datosh
  • 498
  • 7
  • 20
  • 1
    Hi, I have found a [feature request](https://issuetracker.google.com/137517429) similar to your issue, you might take a look and feel free to comment on the feature request to bring more attention. However, I have also found couple of documentations [1](https://docs.fluentd.org/parser#list-of-built-in-parsers) [2](https://cloud.google.com/logging/docs/structured-logging#default-parsers) [3](https://cloud.google.com/logging/docs/setup/nodejs) [4](https://cloud.google.com/logging/docs/structured-logging#structured-log-install%20h), these might can help you to get a better idea. – Aarti S Sep 17 '20 at 00:19

0 Answers0