0

I am trying to attach a logging handler to the Python root logger in my cloud function. I am using below code :

    import logging
    import os
    import google.cloud.logging
    from google.cloud.logging.handlers import CloudLoggingHandler, setup_logging
    
    logLabels = {'ServerName' : os.uname().nodename}
    logging_client = google.cloud.logging.Client()
    handler = CloudLoggingHandler(logging_client,name='my-test-cf',labels=logLabels)
    logging.getLogger().setLevel(logging.INFO)
    setup_logging(handler, excluded_loggers=('google.cloud', 'google.auth', 'google_auth_httplib2'))
    logging.info(f"Starting Execution of My Test CF")

Result is duplicate log entries one with info and other with error

  1. info -> Starting Execution of My Test CF
  2. error -> Starting Execution of My Test CF

Any suggestions would be of great help. Thank you. enter image description here

thakur babban
  • 33
  • 1
  • 7

1 Answers1

2

First you need to setup the logging to sent all log intries to Cloud Logging by attaching the Cloud Logging handler to the Python Root logger.

import google.cloud.logging
client = google.cloud.logging.Client()
client.get_default_handler()
client.setup_logging()

Once the handler is attached, any logs (by default INFO) which are emitted in you application will be sent to Logging.

import logging
logging.info(f"Starting Execution of My Test CF")
Vikram Shinde
  • 1,022
  • 6
  • 17
  • Logs are being sent to Logging. Problem is that it make duplicate entries. – thakur babban Aug 04 '20 at 10:59
  • Put 'import logging' statement after setting up logging handler – Vikram Shinde Aug 05 '20 at 05:50
  • I used below code def start_process(request): import google.cloud.logging client = google.cloud.logging.Client() client.get_default_handler() client.setup_logging() import logging logging.info(f"Starting Execution of My Test CF") but result is duplicate entries D 2020-08-05T06:50:27.166180883Z Function execution started I 2020-08-05T06:50:28.167Z Starting Execution of My Test CF E 2020-08-05T06:50:28.167Z Starting Execution of My Test CF D 2020-08-05T06:50:28.168703870Z Function execution took 1003 ms, finished with status code: 200 – thakur babban Aug 05 '20 at 06:57
  • The duplicate entries can be from 1. Normal app logging into stdout. 2. Stackdriver loggings Use filter on stackdriver with selecting appropriate app – Vikram Shinde Aug 05 '20 at 06:59
  • When I am removing handler code then it does not result duplicate entries. like below import logging logging.info(f"Starting Execution of My Test CF") then result is 2020-08-05 00:08:13.256 PDT Starting Execution of My Test CF – thakur babban Aug 05 '20 at 07:09
  • Do you want to share screenshot of duplicate entries. – Vikram Shinde Aug 05 '20 at 08:04
  • sure. I added screen-print of stackdriver log in question. – thakur babban Aug 05 '20 at 08:34
  • I think, Google has changed the logging implementation in Cloud Function. No need to setup anything. just `import logging` and it will work. – Vikram Shinde Aug 05 '20 at 09:30
  • I understand that the code used, as outlined by the google public documentation to [connect Cloud Logging and the Python native logging module](https://cloud.google.com/logging/docs/setup/python#connecting_the_library_to_python_logging), is causing duplicate entries in your Cloud Functions logs. It is also suspected that Cloud Functions automatically integrates Cloud Logging with the Python logging module. As of now I am in the process of verifying if this is the case with our Cloud Functions specialists. Please expect a update by August 9 or as soon as there are details made available. – KevinH Aug 09 '20 at 20:15
  • It seems that a similar issue was found on another [Stack Overflow post](https://stackoverflow.com/questions/48078051/duplicate-log-entries-with-google-cloud-stackdriver-logging-of-python-code-on-ku/58655297#58655297). Please attempt the workarounds provided here and see if the issue persists for the time being. I will provide a update here as soon as details are made available from our specialists. – KevinH Aug 11 '20 at 03:21
  • It seems that this issue is indeed reproducible and this functionality is not mentioned particularly anywhere on our public documentation. Therefore it is recommended that this issue should be investigated further. Firstly a possible workaround for this issue would be to utilize the [Cloud Logging client libraries](https://cloud.google.com/functions/docs/monitoring/logging#using_the_logging_api) so as to write logs to Cloud Logging. Secondly, please submit a new [Public Tracker](https://developers.google.com/issue-tracker/) on Public Issue Tracker so that we may investigate this issue further. – KevinH Aug 15 '20 at 23:02