1

I'm struggling to find out why my log entries are being duplicated in Cloud Logging. I use a custom dummy handler that does nothing, and I'm also using a named logger.

Here's my code:

import google.cloud.logging
import logging


class MyLogHandler(logging.StreamHandler):
    def emit(self, record):
        pass

# Setting Up App Engine's Logging
client = google.cloud.logging.Client()
client.get_default_handler()
client.setup_logging()

# Setting Up my custom logger/handler
my_handler = MyLogHandler()
logging.getLogger('my_logger').addHandler(my_handler)
logging.getLogger('my_logger').setLevel(logging.DEBUG)

logging.getLogger('my_logger').debug('Why this message is being duplicated?') # please note that i'm logging into 'my_logger' logger, I'm not using root logger for this message

In the first place, I think this message shouldn't even show in Cloud Logging because i'm using a named logger called 'my_logger' and cloud logging is attached to root logger only, but anyway...

The above code is imported into my app.py which bootstraps a Flask app on app engine.

Here is a screenshot of the issue:

Duplicated Messages

This guy has a similar issue: Duplicate log entries with Google Cloud Stackdriver logging of Python code on Kubernetes Engine

But I tried every workaround suggested in that topic and didn't work either. Is there something I'm missing here? Thanks in advance.

Thiago Hencke
  • 301
  • 2
  • 11
  • If you add a handler to a sub-logger the message is still send upwards to the root logger and handled again unless you set the `propagate` attribute of the sub-logger to `False`. – Klaus D. Jan 27 '21 at 04:51
  • Hi, already tried that, set propagate = False works locally but somehow when deployed to app engine it does not work, still duplicates the records. The duplicated records are emitted by the same logger (can confirm this viewing the jsonPayload/python_logger property) – Thiago Hencke Jan 27 '21 at 20:15
  • Make sure that `client.setup_logging()` is only called once. It will add a new handler every time it is called. This happened to me once, though I can't say if that is the same issue you are having without more context. – Zack Jan 29 '21 at 03:40
  • As per the Official Google Cloud Platform [documentation](https://cloud.google.com/logging/docs/setup/python#run-gae) Cloud Logging is enabled automatically for App Engine. Therefore it is possible that your duplicate logs are caused by both the Cloud Logging and the Cloud Logging library for Python. Could you expand the log information to determine if both log entries have the same insertID? – Dylan Feb 17 '21 at 11:54

1 Answers1

0
from google.cloud.logging.handlers import AppEngineHandler    

root_logger = logging.getLogger()

# use the GCP appengine handlers only in order to prevent logs from getting written to STDERR
root_logger.handlers = [handler for handler in root_logger.handlers
                        if isinstance(handler, AppEngineHandler)]
Utkarsh Shende
  • 11
  • 3