0

Background

I am trying to write logs in Django that not only stores the messages but also necessary information of the requests like IP address and user in the logs.

I noticed that I could use the extra attribute of logger to include request like logger.info("my message", extra={'ip': request.ip, 'user': request.user}. But this will be extremely cumbersome to include this extra block for every log in my app. It will also be unmanageable if I wish to add an extra attribute to display in the future.

Based on my understanding LogAdapter is only useful when the extra context is static.

I tried to follow How can I add the currently logged in username to the access log of django.server? to use a custom middleware to store request object in thread local storage and read the object from my custom filter.

import logging
import threading

local = threading.local()

class RequestFilter(logging.Filter):
    def filter(self, record):
        request = getattr(local, 'request', None)
        if request:
            record.ip = request.META.get('REMOTE_ADDR')
            record.user = request.user.username
        else:
            record.ip = '-'
            record.user = '-'
        return True


class LogRequestMiddleware:
    """
    A middleware that stores request object into the thread local storage,
    so that logging filter can access it and populate necessary information to the logs
    """

    def __init__(self, get_response):
        self.get_response = get_response

    def __call__(self, request):
        setattr(local, 'request', request)
        return self.get_response(request)

Question

  1. thread.local method worked but I wonder if this approach is safe in production as I am using nginx+gunicorn+django in production. Is there a potential risk that the thread local storage may be overwritten by other treads?

  2. I have the following LOGGING configuration, which writes the log to a file. Will there be a race condition that multiple threads tries to write to the file the same time?

  3. I believe my question should be a common one in django development, but I can hardly find any resource on this question. I wonder if there is a better approach of doing this. Thanks ahead!


LOGGING = {
    'version': 1,
    'disable_existing_loggers': False,
    'formatters': {
        "simple": {
            "format": "{levelname} {asctime} {ip} {funcName} {user} {message}",
            "style": "{",
        },
    },
    'filters': {
        'ip_filter': {
            '()': 'EngMemo.logging.RequestFilter',
        }
    },
    'handlers': {
        'file': {
            'level': 'INFO',
            'class': 'logging.handlers.RotatingFileHandler',
            'filename': LOGS_DIR + 'api.log',
            'maxBytes': 1024 * 1024,  # 1 MB
            'backupCount': 5,  # Number of backup log files to keep
            'formatter': 'simple',
            'filters': ['ip_filter'],
        },
    },
    'loggers': {
        'api_log': {
            'handlers': ['file'],
            'level': 'INFO',
            'propagate': True,
        },
    },
}
Hao
  • 65
  • 6

0 Answers0