Can Python's logging format be modified depending on the message log level?

Question

I'm using Python's logging mechanism to print output to the screen. I could do this with print statements, but I want to allow a finer-tuned granularity for the user to disable certain types of output. I like the format printed for errors, but would prefer a simpler format when the output level is "info."

For example:

  logger.error("Running cmd failed")
  logger.info("Running cmd passed")

In this example, I would like the format of the error to be printed differently:

# error
Aug 27, 2009 - ERROR: Running cmd failed
# info
Running cmd passed

Is it possible to have different formats for different log levels without having multiple logging objects? I'd prefer to do this without modifying the logger once it's created since there are a high number of if/else statements to determine how the output should be logged.

score 92 · Answer 1 · edited May 23 '17 at 11:54

92

I just ran into this issue and had trouble filling in the "holes" left in the above example. Here's a more complete, working version that I used. Hopefully this helps someone:

# Custom formatter
class MyFormatter(logging.Formatter):

    err_fmt  = "ERROR: %(msg)s"
    dbg_fmt  = "DBG: %(module)s: %(lineno)d: %(msg)s"
    info_fmt = "%(msg)s"


    def __init__(self, fmt="%(levelno)s: %(msg)s"):
        logging.Formatter.__init__(self, fmt)


    def format(self, record):

        # Save the original format configured by the user
        # when the logger formatter was instantiated
        format_orig = self._fmt

        # Replace the original format with one customized by logging level
        if record.levelno == logging.DEBUG:
            self._fmt = MyFormatter.dbg_fmt

        elif record.levelno == logging.INFO:
            self._fmt = MyFormatter.info_fmt

        elif record.levelno == logging.ERROR:
            self._fmt = MyFormatter.err_fmt

        # Call the original formatter class to do the grunt work
        result = logging.Formatter.format(self, record)

        # Restore the original format configured by the user
        self._fmt = format_orig

        return result

Edit:

Compliments of Halloleo, here's an example of how to use the above in your script:

fmt = MyFormatter()
hdlr = logging.StreamHandler(sys.stdout)

hdlr.setFormatter(fmt)
logging.root.addHandler(hdlr)
logging.root.setLevel(DEBUG)

Edit 2:

Python3 logging has changed a bit. See here for a Python3 approach.

edited May 23 '17 at 11:54

Community

1
1

answered Dec 01 '11 at 22:15

JS.

14,781
13
63
75

This works great! I changed the references to `MyFormatter` by name to `self` for more consistency. – halloleo Sep 06 '12 at 04:22
2

And here I might add the way you can use the MyFormatter class in your program (replace each with a carriage return): `fmt = MyFormatter()` `hdlr = logging.StreamHandler(sys.stdout)` ` `hdlr.setFormatter(fmt)` `logging.root.addHandler(hdlr)` `logging.root.setLevel(DEBUG)` – halloleo Sep 06 '12 at 04:25
7

This answer won't work after 3.2, due to the changes in internal logging mechanism. `logging.Formatter.format` now depends of the `style` parameter of `__init__`. – Evpok May 21 '13 at 00:32
3

Evpok is right. Add this after you assign self._fmt: `self._style = logging.PercentStyle(self._fmt)` – Ross R Oct 11 '14 at 12:55
Better use `%(message)` instead of `%(msg)`, otherwise you can't use format strings (e.g. `logger.info("foo %s", "bar")`) – luator Aug 26 '15 at 14:46
3

Could this be improved by using `super()` instead of calling `logging.Formatter`? – phoenix Nov 16 '15 at 16:31
1

@phoenix: Yes. Good observation. – JS. Dec 15 '15 at 00:26
@Evpok your comment did save me a lot of effort. Cheers!! – Vinay Feb 09 '17 at 06:05
Using this, the line number that gets logged is the line number of the call to logger, not the line of the error in question. Any work around for this? – Nishant Roy Jul 05 '17 at 23:17
how can I provide time format here ? generally we format for time like this `formatter = logging.Formatter("[%(asctime)s] -[%(levelname)] - %(message)s","%d-%m-%Y %I:%M:%S %p")` – Eular Aug 25 '19 at 06:17
Do you have an idea why the `__init__` function is not called when I use a custom formatter with `dictConfig` and pass it via `()`? – t3chb0t Jan 27 '23 at 15:07

score 33 · Accepted Answer · answered Aug 27 '09 at 19:20

33

Yes, you can do this by having a custom Formatter class:

class MyFormatter(logging.Formatter):
    def format(self, record):
        #compute s according to record.levelno
        #for example, by setting self._fmt
        #according to the levelno, then calling
        #the superclass to do the actual formatting
        return s

Then attach a MyFormatter instance to your handlers.

answered Aug 27 '09 at 19:20

Vinay Sajip

95,872
14
179
191

Outstanding - that worked perfectly. I modified the format() method to check levelno and change the message if need be. Otherwise it resets it back to the original string I passed in. Thanks! – bedwyr Aug 27 '09 at 20:10
14

Please remove the checkmark from this answer. The one just below this is complete. There are wide swaths of code missing from this answer where there are just comments describing what you *should* do. – Utkonos Jun 04 '17 at 08:19

score 20 · Answer 3 · answered Jul 09 '19 at 01:30

ONE WAY OF DOING THIS

Define a class

import logging

class CustomFormatter(logging.Formatter):
    """Logging Formatter to add colors and count warning / errors"""

    grey = "\x1b[38;21m"
    yellow = "\x1b[33;21m"
    red = "\x1b[31;21m"
    bold_red = "\x1b[31;1m"
    reset = "\x1b[0m"
    format = "%(asctime)s - %(name)s - %(levelname)s - %(message)s (%(filename)s:%(lineno)d)"

    FORMATS = {
        logging.DEBUG: grey + format + reset,
        logging.INFO: grey + format + reset,
        logging.WARNING: yellow + format + reset,
        logging.ERROR: red + format + reset,
        logging.CRITICAL: bold_red + format + reset
    }

    def format(self, record):
        log_fmt = self.FORMATS.get(record.levelno)
        formatter = logging.Formatter(log_fmt)
        return formatter.format(record)

Instantiate logger

# create logger with 'spam_application'
logger = logging.getLogger("My_app")
logger.setLevel(logging.DEBUG)

# create console handler with a higher log level
ch = logging.StreamHandler()
ch.setLevel(logging.DEBUG)

ch.setFormatter(CustomFormatter())

logger.addHandler(ch)

And use!

logger.debug("debug message")
logger.info("info message")
logger.warning("warning message")
logger.error("error message")
logger.critical("critical message")

Result

I try to keep the same answer up to date here https://stackoverflow.com/a/56944256/9150146, if helps please upvote. Thank you @user3821178 — Sergey Pleshakov, Mar 25 '20 at 12:16
This is an excellent answer. Colour coding logging is also very helpful. — Akash Desarda, Aug 09 '20 at 12:47
Note that you can instantiate the formatters once in the init instead of on every call to format — postelrich, Mar 30 '22 at 14:42

score 17 · Answer 4 · answered Jan 25 '13 at 10:44

And again like JS answer but more compact.

class SpecialFormatter(logging.Formatter):
    FORMATS = {logging.DEBUG :"DBG: %(module)s: %(lineno)d: %(message)s",
               logging.ERROR : "ERROR: %(message)s",
               logging.INFO : "%(message)s",
               'DEFAULT' : "%(levelname)s: %(message)s"}

    def format(self, record):
        self._fmt = self.FORMATS.get(record.levelno, self.FORMATS['DEFAULT'])
        return logging.Formatter.format(self, record)

hdlr = logging.StreamHandler(sys.stderr)
hdlr.setFormatter(SpecialFormatter())
logging.root.addHandler(hdlr)
logging.root.setLevel(logging.INFO)

This answer won't work after 3.2, due to the changes in internal logging mechanism. `logging.Formatter.format` now depends of the `style` parameter of `__init__`. — Evpok, May 21 '13 at 00:33

score 14 · Answer 5 · answered Jan 08 '15 at 07:51

Instead of relying on styles or internal fields, you could also create a Formatter that delegates to other formatters depending on record.levelno (or other criteria). This is a slightly cleaner solution in my humble opinion. Code below should work for any python version >= 2.7:

The simple way would look something like this:

class MyFormatter(logging.Formatter):

    default_fmt = logging.Formatter('%(levelname)s in %(name)s: %(message)s')
    info_fmt = logging.Formatter('%(message)s')

    def format(self, record):
        if record.levelno == logging.INFO:
            return self.info_fmt.format(record)
        else:
            return self.default_fmt.format(record)

But you could make it more generic:

class VarFormatter(logging.Formatter):

    default_formatter = logging.Formatter('%(levelname)s in %(name)s: %(message)s')

    def __init__(self, formats):
        """ formats is a dict { loglevel : logformat } """
        self.formatters = {}
        for loglevel in formats:
            self.formatters[loglevel] = logging.Formatter(formats[loglevel])

    def format(self, record):
        formatter = self.formatters.get(record.levelno, self.default_formatter)
        return formatter.format(record)

I used a dict as input here, but obviously you could also use tuples, **kwargs, whatever floats your boat. This would then be used like:

formatter = VarFormatter({logging.INFO: '[%(message)s]', 
                          logging.WARNING: 'warning: %(message)s'})
<... attach formatter to logger ...>

score 10 · Answer 6 · edited May 23 '17 at 11:54

This is an adaptation of estani's answer to the new implementation of logging.Formatter which now relies on formatting styles. Mine relies on '{' style format, but it can be adapted. Could be refined to be more general and allow selection of formatting style and custom messages as arguments to __init__, too.

class SpecialFormatter(logging.Formatter):
    FORMATS = {logging.DEBUG : logging._STYLES['{']("{module} DEBUG: {lineno}: {message}"),
           logging.ERROR : logging._STYLES['{']("{module} ERROR: {message}"),
           logging.INFO : logging._STYLES['{']("{module}: {message}"),
           'DEFAULT' : logging._STYLES['{']("{module}: {message}")}

    def format(self, record):
        # Ugly. Should be better
        self._style = self.FORMATS.get(record.levelno, self.FORMATS['DEFAULT'])
        return logging.Formatter.format(self, record)

hdlr = logging.StreamHandler(sys.stderr)
hdlr.setFormatter(SpecialFormatter())
logging.root.addHandler(hdlr)
logging.root.setLevel(logging.INFO)

Thanks for updating this to work with Python3. I ran into the same problem in Python3 and came up with a similar solution. Would you please be so kind as to post this answer there as well? http://stackoverflow.com/questions/14844970/modifying-logging-message-format-based-on-message-logging-level-in-python3 — JS., May 21 '13 at 19:20

user1837990 · Answer 7 · 2014-02-22T19:09:20.227

The above solution works with 3.3.3 release. However with 3.3.4 you get the following error.

FORMATS = { logging.DEBUG : logging._STYLES['{']("{module} DEBUG: {lineno}: {message}"),

TypeError: 'tuple' object is not callable

After some searching around in the logging class Lib\logging__init__.py I found that a data structure has changed from 3.3.3 to 3.3.4 that causes the issue

3.3.3

_STYLES = {
    '%': PercentStyle,
    '{': StrFormatStyle,
    '$': StringTemplateStyle
}

3.3.4

_STYLES = {
   '%': (PercentStyle, BASIC_FORMAT),
   '{': (StrFormatStyle, '{levelname}:{name}:{message} AA'),
    '$': (StringTemplateStyle, '${levelname}:${name}:${message} BB'),
}

The updated solution is therefore

class SpecialFormatter(logging.Formatter):
     FORMATS = {logging.DEBUG : logging._STYLES['{'][0]("{module} DEBUG: {lineno}: {message}"),
       logging.ERROR : logging._STYLES['{'][0]("{module} ERROR: {message}"),
       logging.INFO : logging._STYLES['{'][0]("{module}: {message}"),
       'DEFAULT' : logging._STYLES['{'][0]("{module}: {message}")}

 def format(self, record):
    # Ugly. Should be better
    self._style = self.FORMATS.get(record.levelno, self.FORMATS['DEFAULT'])
    return logging.Formatter.format(self, record)

It may be easier to just import the style type directly with `from logging import StrFormatStyle` instead of doing `logging._STYLES['{'][0]` — TomDotTom, May 02 '19 at 11:54

score 5 · Answer 8 · answered Aug 03 '14 at 04:46

If you are just looking to skip formatting certain levels, you can do something simpler than the other answers like the following:

class FormatterNotFormattingInfo(logging.Formatter):
    def __init__(self, fmt = '%(levelname)s:%(message)s'):
        logging.Formatter.__init__(self, fmt)

    def format(self, record):
        if record.levelno == logging.INFO:
            return record.getMessage()
        return logging.Formatter.format(self, record)

This also has the advantage of working before and after the 3.2 release by not using internal variables like self._fmt nor self._style.

I think this is most clean solution – vlk Nov 09 '17 at 19:03 — vlk, Nov 09 '17 at 19:03

Can Python's logging format be modified depending on the message log level?

8 Answers8

ONE WAY OF DOING THIS

3.3.3

3.3.4

Linked

Related