How many times was logging.error() called?

Question

Maybe it's just doesn't exist, as I cannot find it. But using python's logging package, is there a way to query a Logger to find out how many times a particular function was called? For example, how many errors/warnings were reported?

Answer 1

The logging module doesn't appear to support this. In the long run you'd probably be better off creating a new module, and adding this feature via sub-classing the items in the existing logging module to add the features you need, but you could also achieve this behavior pretty easily with a decorator:

class CallCounted:
    """Decorator to determine number of calls for a method"""

    def __init__(self,method):
        self.method=method
        self.counter=0

    def __call__(self,*args,**kwargs):
        self.counter+=1
        return self.method(*args,**kwargs)


import logging
logging.error = CallCounted(logging.error)
logging.error('one')
logging.error('two')
print(logging.error.counter)

Output:

ERROR:root:one
ERROR:root:two
2

Answer 2

You can also add a new Handler to the logger which counts all calls:

class MsgCounterHandler(logging.Handler):
    level2count = None

    def __init__(self, *args, **kwargs):
        super(MsgCounterHandler, self).__init__(*args, **kwargs)
        self.level2count = {}

    def emit(self, record):
        l = record.levelname
        if (l not in self.level2count):
            self.level2count[l] = 0
        self.level2count[l] += 1

You can then use the dict afterwards to output the number of calls.

Answer 3

There'a a warnings module that -- to an extent -- does some of that.

You might want to add this counting feature to a customized Handler. The problem is that there are a million handlers and you might want to add it to more than one kind.

You might want to add it to a Filter, since that's independent of the Handlers in use.

Answer 4

Based on Rolf's answer and how to write a dictionary to a file, here another solution which stores the counts in a json file. In case the json file exists and continue_counts=True, it restores the counts on initialisation.

import json
import logging
import logging.handlers
import os


class MsgCounterHandler(logging.Handler):
    """
    A handler class which counts the logging records by level and periodically writes the counts to a json file.
    """

    level2count_dict = None

    def __init__(self, filename, continue_counts=True, *args, **kwargs):
        """
        Initialize the handler.

        PARAMETER
        ---------
        continue_counts: bool, optional
            defines if the counts should be loaded and restored if the json file exists already.
        """
        super(MsgCounterHandler, self).__init__(*args, **kwargs)

        filename = os.fspath(filename)
        self.baseFilename = os.path.abspath(filename)

        self.continue_counts = continue_counts

        # if another instance of this class is created, get the actual counts
        if self.level2count_dict is None:
            self.level2count_dict = self.load_counts_from_file()

    def emit(self, record):
        """
        Counts a record.
        In case, create add the level to the dict.
        If the time has come, update the json file.
        """
        level = record.levelname
        if level not in self.level2count_dict:
            self.level2count_dict[level] = 0
        self.level2count_dict[level] += 1

        self.flush()

    def flush(self):
        """
        Flushes the dictionary.
        """
        self.acquire()
        try:
            json.dump(self.level2count_dict, open(self.baseFilename, 'w'))
        finally:
            self.release()

    def load_counts_from_file(self):
        """
        Load the dictionary from a json file or create an empty dictionary
        """
        if os.path.exists(self.baseFilename) and self.continue_counts:
            try:
                level2count_dict = json.load(open(self.baseFilename))
            except Exception as a:
                logging.warning(f'Failed to load counts with: {a}')
                level2count_dict = {}
        else:
            level2count_dict = {}
        return level2count_dict