Django rest framework logging different levels on different files

Question

I am working on Django REST framework and I want to have separate files for logging data.

I want to have a file for the simple transactions e.g. GET, PUT, POST etc. and one file with the errors that I will collect in case of an error.

I have been reading the Django Logging Documentation and I came up with some configurations on how to log the info data. Sample of configurations bellow:

settings.py

STATIC_URL = '/static/'
PROJECT_ROOT = os.path.dirname(os.path.abspath(__file__))
STATIC_ROOT = os.path.join(PROJECT_ROOT, 'static')
LOGGING_ROOT = os.path.join(STATIC_ROOT, 'logging')

LOGGING = {
    'version': 1,
    'disable_existing_loggers': False,
    'handlers': {
        'file': {
            'level': 'INFO',
            'class': 'logging.FileHandler',
            'filename': LOGGING_ROOT + "/info.log",
        },
    },
    'loggers': {
        'django': {
            'handlers': ['file'],
            'level': 'INFO',
            'propagate': True,
        },
    },
}

It works as expected sample of data in file:

"PUT /upload/dat.txt HTTP/1.1" 204 14
"OPTIONS / HTTP/1.1" 200 10020
"GET / HTTP/1.1" 200 9916

I tried to apply another handler in the settings.py file e.g.:

LOGGING = {
    'version': 1,
    'disable_existing_loggers': False,
    'handlers': {
        'fileInfo': {
            'level': 'INFO',
            'class': 'logging.FileHandler',
            'filename': LOGGING_ROOT + "/info.log",
        },
        'fileDebug': {
            'level': 'DEBUG',
            'class': 'logging.FileHandler',
            'filename': LOGGING_ROOT + "/debbug.log",
        },
    },
    'loggers': {
        'django': {
            'handlers': ['fileInfo'],
            'level': 'INFO',
            'propagate': True,
        },
        'django': {
            'handlers': ['fileDebug'],
            'level': 'DEBUG',
            'propagate': True,
        },
    },
}

I does not work as expected, now I am getting all the data (INFO and DEBUG) on the same file debug.log. So I decided to have another approach through the logging library. What I can not figure out is how to pipe the output of these log errors into the directory e.g. (path/errors.log).

From the documentation:

# import the logging library
import logging

# Get an instance of a logger
logger = logging.getLogger(__name__)

def my_view(request, arg1, arg):
    ...
    if bad_mojo:
        # Log an error message
        logger.error('Something went wrong!')

I found this similar question No handlers could be found for logger and it works if I apply the following:

import logging
logging.basicConfig()
logger = logging.getLogger(__name__)
logger.error('Something went wrong!')

Now everything is logged on the stdout (INFO and ERROR). Is there a way to combine those configurations?

I want to log all requests into the info.log file, and log errors in the error.log file when I choose on my file.

Update:

Solution provided by bruno desthuilliers sample bellow:

settings.py

LOGGING = {
    'version': 1,
    'disable_existing_loggers': False,
    'handlers': {
        'fileInfo': {
            'level': 'INFO',
            'class': 'logging.FileHandler',
            'filename': os.path.join(LOGGING_ROOT, "info.log"),
        },
        'fileDebug': {
            'level': 'DEBUG',
            'class': 'logging.FileHandler',
            'filename': os.path.join(LOGGING_ROOT, "debug.log")
        },
        'console': {
            'class': 'logging.StreamHandler',
        },
    },
    'loggers': {
        'django': {
            'handlers': ['console', 'fileInfo', 'fileDebug'],
            'level': 'DEBUG',
            'propagate': True,
        },
    },
}

I also found Making Python loggers output all messages to stdout in addition to log. Sample of my current modifications:

import logging
from thanosTest import settings
logging.basicConfig(filename=os.path.join(settings.LOGGING_ROOT, "error.log"))
stderrLogger = logging.StreamHandler()
stderrLogger.setFormatter(logging.Formatter(logging.BASIC_FORMAT))
logging.getLogger().addHandler(stderrLogger)
logging.error('Something went wrong!')

Now everything gets logged on the error.log. I guess I need to apply some filtering or something. I will try to figure it out.

Have a look into logging levels in python, Logging messages which are less severe than level will be ignored; logging messages which have severity level or higher will be emitted by whichever handler or handlers service this logger, unless a handler’s level has been set to a higher severity level than level. Refer : https://docs.python.org/3/library/logging.html#levels In the case mentioned above : Debug file will also have Info logs, but Info file wont have Debug logs — Pratik Charwad, Jul 26 '19 at 13:36

score 9 · Accepted Answer · answered Mar 07 '18 at 11:41

Here:

'loggers': {
    'django': {
        'handlers': ['fileInfo'],
        'level': 'INFO',
        'propagate': True,
    },
    'django': {
        'handlers': ['fileDebug'],
        'level': 'DEBUG',
        'propagate': True,
    },

You define the 'django' key twice, so the second overwrite the first.

As a general rule, if you want specific settings for a given logger, configure it as a distinct logger (ie one logger per package or django app). Also note that the logger's "handlers" are lists so you can have more than one handler per logger (ie one for debug and one for info). The logging lib is a bit complex but taking time to read the full doc and experimenting a bit is worth the effort, really.

A couple other notes:

using file handlers for multiprocess apps (in production django is most often served by multiprocess front servers) is more often than not a bad idea (concurrent write accesses to a file never really work).
this 'filename': LOGGING_ROOT + "/info.log" kind of defeats the whole point of using os.path - you want os.path.join(LOGGING_ROOT, "info.log") instead

it works as expected, but I am still having a minor problem that I can not resolve with logger.error('Error message'). I will update my question to provide more information and findings. — Thanos, Mar 07 '18 at 12:06

Django rest framework logging different levels on different files

1 Answers1

Linked