0

I am running Django application on Google App Engine standard env. I have two services running for dev and prod. both services plus my local machine are configured with Google Logging Client for logging. Recently, for some reason, the prod service stopped posting logs into StackDriver. But both my local and dev service are posting logs.

here is my code:

settings.py

    from google.cloud import logging 
    if os.getenv('GAE_APPLICATION', None):
        client = logging.Client()
    else:
       # setup for my local logging.
       client = logging.Client.from_service_account_json('PATH TO JSON FILE')
     
    LOGGING = {
    'version': 1,
    'disable_existing_loggers': True,
    'formatters': {
        'verbose': {
            'format': '%(levelname)s %(asctime)s %(module)s %(process)d %(thread)d %(message)s'
        },
        'simple': {
            'format': '%(levelname)s %(message)s'
        },
    },
    'handlers': {
        'stackdriver': {
            'level': 'INFO',
            'class': 'google.cloud.logging.handlers.CloudLoggingHandler',
            'client': client
        },
        'console':{
            'level': 'INFO',
            'class': 'logging.StreamHandler',
            'formatter': 'simple'
        },
        'mail_admins': {
            'level': 'CRITICAL',
            'class': 'django.utils.log.AdminEmailHandler',
            "email_backend": "anymail.backends.mailgun.EmailBackend"
        }
    },
    'loggers': {
        'django': {
            'handlers': ['stackdriver'],
            'propagate': True,
            'level': 'INFO',
        },
        'django.request': {
            'handlers': ['stackdriver', 'console', 'mail_admins'],
            'level': 'INFO',
            'propagate': True,
        },
    } }

urls.py

    def trigger_error(request):
        import logging
        logger = logging.getLogger('stackdriver')
        logger.error('test error')
        logger.critical('test critical')
        logger.warning('test warning')
        logger.info('test info') 
        division_by_zero = 1 / 0
    

    path('debug/', trigger_error),

My questions are:

  1. Is there any limits or quotes for logging into a specific service? if so, how to check if my prod service exhausted these limits?
  2. Is there anything else I can do to debug this issue?

thanks!

EDIT:

I made some progress. I changed the client and handler to logging_v2, added root and empty loggers, and removed an old logger no longer exists.

if os.getenv('GAE_APPLICATION', None): 
    client = google.cloud.logging_v2.Client()
else:
    client = google.cloud.logging_v2.Client.from_service_account_json('PATH TO JSON FILE')
client.setup_logging()


LOGGING = {
'version': 1,
'disable_existing_loggers': True,
'formatters': {
    'verbose': {
        'format': '%(levelname)s %(asctime)s %(module)s %(process)d %(thread)d %(message)s'
    },
    'simple': {
        'format': '%(levelname)s %(message)s'
    },
},
'handlers': {
    'stackdriver': {
        'level': 'INFO',
        'class': 'google.cloud.logging_v2.handlers.CloudLoggingHandler',
        'client': client,
        'formatter': 'verbose'
    },
    'console':{
        'level': 'INFO',
        'class': 'logging.StreamHandler',
        'formatter': 'simple'
    }
},
'loggers': {
    '': {
        'level': 'INFO',
        'handlers': ['stackdriver', 'console'],
        'propagate': True
    },
    'root': {
        'level': 'INFO',
        'handlers': ['stackdriver', 'console'],
        'propagate': True
    },
    'django': {
        'handlers': ['stackdriver', 'console'],
        'propagate': True,
        'level': 'INFO'
    },
    'django.request': {
        'handlers': ['stackdriver', 'console'],
        'level': 'INFO',
        'propagate': True
    },
}
}

now logs are showing up in the service logs but missing severity. enter image description here

RJC
  • 1,224
  • 2
  • 12
Zeedia
  • 1,313
  • 14
  • 20
  • To answer your questions; 1. This [link](https://cloud.google.com/logging/quotas#api-limits) discusses the quotas and limitation of the Cloud Logging; 2. Please verify if you've updated your prod environment and check if it has the same version with your dev and stage environment. – RJC Dec 22 '21 at 07:43
  • Thanks @RJC I checked the quotes and the service is away from reaching the logging limits. for your second request do you mean updating the google logging package? or something else? – Zeedia Dec 22 '21 at 18:31
  • 1
    For my second question, did you change or updated the libraries in your prod environment? Additionally, you can check this [trouble shooting guide](https://cloud.google.com/logging/docs/agent/logging/troubleshooting#common-issues). – RJC Dec 23 '21 at 02:49
  • @RJC yeah, I think one old logger package was causing the problem. now logs are showing up but without severity. I updated the question with the latest progress. – Zeedia Dec 23 '21 at 18:51
  • There's a related [StackOverflow case](https://stackoverflow.com/a/65556982/16531380) that stated there's an opened case on [issue tracker](https://issuetracker.google.com/176570488). Additionally, there's also a related [StackOverflow on StackDriver Logging](https://stackoverflow.com/q/56807817/16531380) where the logging severity have duplication issue, and here's a [Github issue link](https://github.com/googleapis/python-logging/issues/91) that further discusses this issue. – RJC Dec 24 '21 at 03:32

0 Answers0