Python logging creates empty log file

Question

I am trying to setup a logging configuration to use in my different modules. I have followed different tutorials and stackoverflow posts (here, here and here) to write logs in to a project.log file.

While the information is displayed correctly in the console, the log.conf is read correctly, the project.log is created but is not filled with the warning messages.

Here is how I proceeded:

The log.conf file used to write up the handlers and formatting:

[loggers]
keys=root,sLogger

[handlers]
keys=consoleHandler,fileHandler

[formatters]
keys=fileFormatter,consoleFormatter

[logger_root]
level=DEBUG
handlers=consoleHandler

[logger_sLogger]
level=DEBUG
handlers=consoleHandler,fileHandler
qualname=sLogger
propagate=0

[handler_consoleHandler]
class=StreamHandler
level=WARNING
formatter=consoleFormatter
args=(sys.stdout,)

[handler_fileHandler]
class=FileHandler
level=WARNING
formatter=fileFormatter
args=('%(logfilename)s', 'w')

[formatter_fileFormatter]
format=%(asctime)s - %(name)s - %(levelname)s - %(message)s

In the main.py file:

import logging
import logging.config

def main():

    logging.config.fileConfig(fname='./log.conf', 
                                defaults={'logfilename': 'project.log')}, 
                                disable_existing_loggers=False)
    logger = logging.getLogger(__name__)

    logger.warning('This is a message')

In my module.py file:

import logging
logger = logging.getLogger(__name__)

logger.warning('Example of a warning message')

score 0 · Accepted Answer · answered Mar 03 '22 at 21:27

I found the problem ! I didn't realize but when defining handlers in the log.conf file, the order of the handlers and formatters is important !

I changed the log.conf changed to:

[loggers]
keys=root,sLogger

[handlers]
keys=consoleHandler,fileHandler

[formatters]
keys=consoleFormatter, fileFormatter

[logger_root]
level=DEBUG
handlers=consoleHandler, fileHandler

[logger_sLogger]
level=DEBUG
handlers=consoleHandler, fileHandler
qualname=sLogger
propagate=0

[handler_consoleHandler]
class=StreamHandler
level=WARNING
formatter=consoleFormatter
args=(sys.stdout,)

[handler_fileHandler]
class=FileHandler
level=WARNING
formatter=fileFormatter
args=('%(logfilename)s','a')

[formatter_fileFormatter]
format=%(asctime)s - %(name)s - %(levelname)s - %(message)s


[formatter_consoleFormatter]
format=%(asctime)s - %(name)s - %(levelname)s - %(message)s

NOTE the changed order between consoleFormatter and fileHandler in the [formatters] section.

The rest is working perfectly fine.

To make it easier to play around with I put it in a function (which is probably redundant I suppose):

def base_logger(path_conf, fname, savedir):
    '''
    Base logging function to be started once
    in the main script then used in every other
    modules.

    Args:
        - path_conf: str, path to the configuration
        file {log.conf}
        - fname: str, name of the saved log file
        - savedir: str, path of the saving directory

    Returns:
        - logging object to be used in the other scripts

    Example:
        In main.py:
            main():
                base_logger(paht_cong = './log.conf', fname='file', savedir='project)
        
        In module.py:
            # AFTER all the import
            logger = logging.getLogger(__name__) # To ge the module name reported in the log file
            ...
            logger.error(f'Error message regarding variable {var}')

        The logger initialization {logger = logging.getLogger(__name__)},
         has to be done in every file that will use logging !

    '''
    logging.config.fileConfig(fname=path_conf,
                                defaults={'logfilename': os.path.join(savedir, f'{fname}.log')}, 
                                disable_existing_loggers=False)

I would add that the logger = logging.getLogger(__name__) has to be placed after all imports to make sure that the module name displayed in the log is not imported from other modules.

Python logging creates empty log file

1 Answers1