How to change the logName and configure the stackdriver logging agent in Container-Optimized OS?

Question

I want to setup an export for logs generated in a Container-Optimized OS using Stackdriver Exports.

In case of Linux VM instance, I know that the logName is taken from the file like /etc/google-fluentd/config.d/[APPLICATION_NAME].conf. For example:
Below is the how /etc/google-fluentd/config.d/syslog.conf looks in a Linux VM:

<source>
  @type tail

  # Parse the timestamp, but still collect the entire line as 'message'
  format syslog

  path /var/log/syslog
  pos_file /var/lib/google-fluentd/pos/syslog.pos
  read_from_head true
  tag some-log-name
</source>

According to the above conf file the logName in StackDriver logs will be "projects/[PROJECT-NAME]/logs/some-log-name". Here is the resource which explains the configuration of Logging Agent, in case where the agent is installed manually.

Now, in the case of CONTAINER-OPTIMIZED OS, there is no folder named /etc/google-fluentd, and I am not able to find out the conf file where I change the logName to reflect in the StackDriver Log Viewer. As of now here is an example of log generated by this VM:

{
   insertId:  "some-random-id"  
   jsonPayload: {…}  
   logName:  "projects/[PROJECT-NAME]/logs/gcplogs-docker-driver"  
   receiveTimestamp:  "2019-03-28T13:10:31.609437487Z"
   resource: {…}  
   timestamp:  "2019-03-28T13:10:30.588317266Z"
}

In the above log, I don't know where gcplogs-docker-driver is coming from in the logName. I am looking for a way to change that.

(P.S. Changing the log name is important for me because, the sinks created to export logs in a google cloud bucket creates a directory whose name is identical to the logName tag (for logName: "projects/[PROJECT-NAME]/logs/gcplogs-docker-driver, a directory called gcplogs-docker-driver will be created.)

score 3 · Answer 1 · answered Nov 06 '19 at 03:46

Now, in the case of CONTAINER-OPTIMIZED OS, there is no folder named /etc/google-fluentd, and I am not able to find out the conf file where I change the logName to reflect in the StackDriver Log Viewer.

The /etc/google-fluentd folder is used to store the config file for the Stackdriver Logging agent (documentation, code).

On Container-optimized OS, the agent is containerized and managed by stackdriver-logging.service, and the config file is stored at /etc/stackdriver/logging.config.d.

You will need to run sudo systemctl start stackdriver-logging to start the agent.

I don't know where gcplogs-docker-driver is coming from in the logName. I am looking for a way to change that.

These logs actually comes from a completely different source. They comes from a Docker logging driver: Google Cloud Logging driver. Looking at the documentation, apparently they do not provide any way to configure the log name.

And looking at their source code, apparently this log name is hard-coded in Docker's source code. So I'm afraid that there is no good way to configure the log name for the gcplogs Docker logging driver.

How to change the logName and configure the stackdriver logging agent in Container-Optimized OS?

1 Answers1