using Python logger class to generate multiple logs for different log levels

Question

I looked through the tutorials for the python logging class here and didnt see anything that would let me make multiple logs of different levels for the same output. In the end I would like to have three logs: <timestamp>_DEBUG.log (debug level)
<timestamp>_INFO.log (info Level)
<timestamp>_ERROR.log (error level)

Is there a way to, in one script, generate multiple log files for the same input?

<-------------UPDATE #1-------------------------->
So in implementing @robert's suggestion, I now have a small issue, probably due to not fully understanding what is being done in his code.

Here is my code in scriptRun.py

import os
import logging

logger = logging.getLogger("exceptionsLogger")
debugLogFileHandler = logging.FileHandler("Debug.log")
errorLogFileHandler = logging.FileHandler("Error.Log")
formatter = logging.Formatter('%(asctime)s %(levelname)s %(message)s')
errorLogFileHandler.setFormatter(formatter)
debugLogFileHandler.setFormatter(formatter)
logger.addHandler(debugLogFileHandler)
logger.addHandler(errorLogFileHandler)

class LevelFilter(logging.Filter):
    def __init__(self, level):
        self.level = level
    def filter(self, record):
        return record.levelno == self.level
debugLogFileHandler.addFilter(LevelFilter(logging.DEBUG))
errorLogFileHandler.addFilter(LevelFilter(logging.ERROR))

directory = []
for dirpath, dirnames, filenames in os.walk("path\to\scripts"):
    for filename in [f for f in filenames if f.endswith(".py")]:
        directory.append(os.path.join(dirpath, filename))
for entry in directory:
    execfile(entry)
    for lists in x:
        if lists[0] == 2:
            logger.error(lists[1]+"   "+lists[2])
        elif lists[0] == 1:
            logger.debug(lists[1]+"   "+lists[2])

an example of what this is running is:

import sys

def script2Test2():
    print y
def script2Ttest3():
    mundo="hungry"

global x 
x = []

theTests = (test2, test3)

for test in theTests:
    try:
        test()
        x.append([1,test.__name__," OK"])
    except:
        error = str(sys.exc_info()[1])
        x.append([2,test.__name__,error])

Now to my issue: running scriptRun.py does not throw any errors when i run it, and error.log and debug.log are created, but only error.log is populated with entries.

any idea why?

<------------------------Update #2----------------------->

So I realized that nothing is being logged that is "lower" than warning. even if i remove the filters and debugLogFileHandler.setLevel(logging.DEBUG) it does not seem to matter. If I set the actual log command to logger.warning or higher, it will print to the logs. Of course once I uncomment debugLogFileHandler.addFilter(LevelFilter(logging.DEBUG)) I get no log activity in Debug.log. I;m tempted to just make my own log level, but that seems like a really bad idea, in case anyone/anything else uses this code.

<-------------------------Final UPDATE--------------------->
Well I was stupid and forgot to set the logger itself to log DEBUG level events. Since by default the logging class doesn't log anything below warning, it wasnt logging any of the debug information I send it.

Final thanks and shoutout to @Robert for the filter.

score 15 · Accepted Answer · answered Sep 16 '11 at 16:12

15

Create multiple Handlers, each for one output file (INFO.log, DEBUG.log etc.).

Add a filter to each handler that only allows the specific level.

For example:

import logging

# Set up loggers and handlers.
# ...

class LevelFilter(logging.Filter):
    def __init__(self, level):
        self.level = level

    def filter(self, record):
        return record.levelno == self.level

debugLogFileHandler.addFilter(LevelFilter(logging.DEBUG))
infoLogFileHandler.addFilter(LevelFilter(logging.INFO))

answered Sep 16 '11 at 16:12

robert

3,484
3
29
38

Just so I understand it better: use just one logger (dont use multiple loggers creating multiple files for different levels), and use this class to filter the log messages each file handler sees? – Snaxib Sep 16 '11 at 16:21
Yes, that's it. Loggers represent areas of your application, handlers represent destinations for logging events, and filters can be applied to either or both to selectively drop events from all or certain destinations. – Vinay Sajip Sep 16 '11 at 16:33
Yes. Of course you could also create multiple loggers, but it is pretty cumbersome to write debugLog.debug("foo") etc. – robert Sep 16 '11 at 16:34
1

could I also so something like this?: `GUILogger = logging.getLogger("guiLogger")` `dh = logging.FileHandler(logFileDEBUG)` `dh.setlevel(logging.DEBUG)` `ih = logging.FileHandler(logFileINFO)` `ih.setlevel(logging.INFO)` `eh = logging.FileHandler(logFileERROR)` `eh.setlevel(logging.ERROR)` `GUILogger.addHandler(dh)` `GUILogger.addHandler(ih)` `GUILogger.addHandler(eh)` – Snaxib Sep 16 '11 at 16:53
1

@Snaxib: With that approach your DEBUG.log, for example, would also contain INFO, WARNING and ERROR messages etc. – robert Sep 16 '11 at 18:35
@robert ok (sorry for the formatting, apprently the formatting stuff for newlines doesnt work on comments?) IM finally getting somewhere with this, in no small part thanks to your help. Thanks a bunch! – Snaxib Sep 16 '11 at 19:45

using Python logger class to generate multiple logs for different log levels

1 Answers1

Linked

Related