How to log only debug messages to a file using Python's logging module and enable debug mode as an option?

Question

I'm rewriting a small daemon from bash to Python-3.x to have more powerful language. I'm really new to Python language.

So, I'm trying to use Python's logging module to log messages in my script. I would like to log all message levels except for debug level through Syslog using SysLogHandlerand logging debug messages to a file if only --debug option is enabled.

I'm using Python-3.6 on Gentoo Gnu/linux. This is for a daemon which auto syncs and auto pretends world update for gentoo's portage package manager. I setup already the logging through Syslog using SysLogHandler and all messages expect for debug ones are shown. I setup as well the logging through the file using WatchedFileHandler, but I haven't found a way to filter only debug messages. Neither I found a way to enable debug only if --debug option is enabled.

My code:

import logging, logging.handlers

debug_log = "debug.log"
debug = "no"

def Create_logger():
    """Setup the logging environment"""
    logging.addLevelName(logging.CRITICAL, '[Crit ]')
    logging.addLevelName(logging.ERROR,    '[Error]')
    logging.addLevelName(logging.WARNING,  '[Warn ]')
    logging.addLevelName(logging.INFO,     '[Info ]')
    logging.addLevelName(logging.DEBUG,    '[Debug]')
    logger           = logging.getLogger(name)
    # Debug part
    file_handler     = logging.handlers.WatchedFileHandler(debug_log)   
    file_handler.setLevel(logging.DEBUG)
    file_formatter   = logging.Formatter('%(asctime)s  %(name)s  %(levelname)s  %(message)s')
    file_handler.setFormatter(file_formatter)
    logger.addHandler(file_handler)
    # Syslog part
    syslog_handler   = logging.handlers.SysLogHandler(address='/dev/log',facility='daemon')
    syslog_handler.setLevel(logging.INFO)
    syslog_formatter = logging.Formatter('%(name)s %(levelname)s %(message)s')
    syslog_handler.setFormatter(syslog_formatter)
    logger.addHandler(syslog_handler)

    return logger

log=Create_logger()
log.error('This is error')

log.setLevel(logging.CRITICAL)

log.error('This is an second error')
log.critical('This is critical !!!')

log.setLevel(logging.INFO)

log.info('Hello world :p')
log.debug('this is an debug message')

log.setLevel(logging.DEBUG)

log.debug(f'This is an other debug message and debug log is locate to {debug_log}')

What I get from /var/log/messages (syslog):

Aug  9 23:43:23 Gentoo syuppod[26195]: [Error] This is error
Aug  9 23:43:23 Gentoo syuppod[26195]: [Crit ] This is critical !!!
Aug  9 23:43:23 Gentoo syuppod[26195]: [Info ] Hello world :p

What I get from debug.log:

2019-08-09 23:43:23,052  syuppod  [Error]  This is error
2019-08-09 23:43:23,052  syuppod  [Crit ]  This is critical !!!
2019-08-09 23:43:23,052  syuppod  [Info ]  Hello world :p
2019-08-09 23:43:23,052  syuppod  [Debug]  This is an other debug message and debug log is locate to debug.log

So, it's Ok for Syslog logging, but not for debug.log file and I tried if debug == "yes" statement before declaring all debug part in function Create_logger(), but it's not working.

I also found this question, but its answer covers only a half of what I'm trying to do. I'm still trying to find a solution in the available documentation.

Could you please advise me?

Jerome Venturi · Accepted Answer · 2019-08-12T06:05:39.847

Ok i found it after reading the available documentation and also : https://stackoverflow.com/a/7447596/190597 (robert) Thx !!

Here is my working code :

#!/usr/bin/env python3
# -*- coding: utf-8 -*-
# -*- python -*- 


import sys, os, argparse
import logging, logging.handlers

__version__ = "0.1-beta1"
name = os.path.basename(os.path.splitext(sys.argv[0])[0])
name_with_ext = os.path.basename(sys.argv[0])
basedir = "/var/lib/" + name
logdir = "/var/log/" + name

# This is for testing only, should go to /var/log/'name'
debug_log = "debug.log"

class LogLevelFilter(logging.Filter):
    """https://stackoverflow.com/a/7447596/190597 (robert)"""
    def __init__(self, level):
        self.level = level

    def filter(self, record):
        # Just revert >= to <= then get only current level or lower.
        return record.levelno <= self.level


def CreateLogger():
    """Setup the logging environment"""
    logging.addLevelName(logging.CRITICAL, '[Crit ]')
    logging.addLevelName(logging.ERROR,    '[Error]')
    logging.addLevelName(logging.WARNING,  '[Warn ]')
    logging.addLevelName(logging.INFO,     '[Info ]')
    logging.addLevelName(logging.DEBUG,    '[Debug]')

    logger           = logging.getLogger(name)

    # File debug only part
    file_handler     = logging.handlers.WatchedFileHandler(debug_log)   
    file_formatter   = logging.Formatter('%(asctime)s  %(name)s  %(levelname)s  %(message)s')
    file_handler.setFormatter(file_formatter)
    file_handler.addFilter(LogLevelFilter(logging.DEBUG))
    file_handler.setLevel(logging.DEBUG)
    logger.addHandler(file_handler)

    # Syslog part
    syslog_handler   = logging.handlers.SysLogHandler(address='/dev/log',facility='daemon')
    syslog_handler.setLevel(logging.INFO)
    syslog_formatter = logging.Formatter('%(name)s %(levelname)s %(message)s')
    syslog_handler.setFormatter(syslog_formatter)
    logger.addHandler(syslog_handler)

    return logger


parser = argparse.ArgumentParser(description='Daemon which automate git update and sync, pretend world update for gentoo portage package manager.',
                                 epilog='Without opts, %(prog)s will start in log level \'info\'')

parser.add_argument('-d', '--debug', help='Start daemon in log level \'debugg\'', action="store_true")
parser.add_argument('-q', '--quiet', help='Start daemon in log level \'quiet\'', action="store_true")
parser.add_argument('-v', '--version', action='version', version='%(prog)s: ' + __version__ + ' - Copyright (C) 2019 Jérôme Venturi, <jerome dot venturi at gmail dot com> - License: GNU/GPL V3.')
args = parser.parse_args()

log=CreateLogger()
# Edited : add log level info or it won't display 'log.info('Hello, how are you?') in /var/log/messages (syslog)
log.setLevel(logging.INFO)
# For Testing
log.debug('This is a debug messages but you should\'nt see it !')
log.info('Hello, how are you ?')
log.error('This is an error !')
log.critical('This is an critial msg and i should exit after this !')

if args.debug:
    log.setLevel(logging.DEBUG)
    log.info(f'Debug log has been enable in {debug_log}')
    log.debug('Debug has been enable.')


log.debug(f'This is an other debug message and debug log is locate to {debug_log}')

How to log only debug messages to a file using Python's logging module and enable debug mode as an option?

1 Answers1