Disable logging per method/function?

Question

I'm new to Python logging and I can easily see how it is preferrable to the home-brew solution I have come up with.

One question I can't seem to find an answer to: how do I squelch logging messages on a per-method/function basis?

My hypothetical module contains a single function. As I develop, the log calls are a great help:

logging.basicConfig(level=logging.DEBUG,
                format=('%(levelname)s: %(funcName)s(): %(message)s'))
log = logging.getLogger()

my_func1():
    stuff...
    log.debug("Here's an interesting value: %r" % some_value)
    log.info("Going great here!")
    more stuff...

As I wrap up my work on 'my_func1' and start work on a second function, 'my_func2', the logging messages from 'my_func1' start going from "helpful" to "clutter".

Is there single-line magic statement, such as 'logging.disabled_in_this_func()' that I can add to the top of 'my_func1' to disable all the logging calls within 'my_func1', but still leave logging calls in all other functions/methods unchanged?

Thanks

linux, Python 2.7.1

Related: http://stackoverflow.com/questions/879732/logging-with-filters — JS., Sep 09 '11 at 02:34

score 15 · Accepted Answer · answered Sep 07 '11 at 23:17

15

The trick is to create multiple loggers.

There are several aspects to this.

First. Don't use logging.basicConfig() at the beginning of a module. Use it only inside the main-import switch

 if __name__ == "__main__":
     logging.basicConfig(...)
     main()
     logging.shutdown()

Second. Never get the "root" logger, except to set global preferences.

Third. Get individual named loggers for things which might be enabled or disabled.

log = logging.getLogger(__name__)

func1_log = logging.getLogger( "{0}.{1}".format( __name__, "my_func1" )

Now you can set logging levels on each named logger.

log.setLevel( logging.INFO )
func1_log.setLevel( logging.ERROR )

answered Sep 07 '11 at 23:17

S.Lott

384,516
81
508
779

Thanks for the recommendation on putting logging configuration inside the main-import switch. Good to know. Is using the root logger not recommended even for a single-module script like I'm working on now? Could you please clarify the cons for doing so? – JS. Sep 08 '11 at 23:28
3

" using the root logger not recommended". Period. Don't use it. It's "anonymous". You want only named loggers so you can configure and filter. – S.Lott Sep 09 '11 at 01:56
1

To elaborate on the clear message of @S.Lott: If you add handlers to the root logger, you will start getting messages from **all** other packages you use, which can get very messy (e.g. `requests`). And vice versa, when another package uses your package, the messages from your package cannot be handled separately either. Bad design. – j08lue Mar 06 '17 at 14:07
1

Combine this answer with [how to get function name as string](https://stackoverflow.com/a/13514318/1483986) for easy, generic copy-paste: `logger = logging.getLogger("{}.{}".format(__name__, sys._getframe().f_code.co_name))` – 7yl4r Mar 22 '18 at 16:21

score 7 · Answer 2 · answered Sep 07 '11 at 22:21

7

You could create a decorator that would temporarily suspend logging, ala:

from functools import wraps

def suspendlogging(func):
    @wraps(func)
    def inner(*args, **kwargs):
        previousloglevel = log.getEffectiveLevel()
        try:
            return func(*args, **kwargs)
        finally:
            log.setLevel(previousloglevel)
    return inner

@suspendlogging
def my_func1(): ...

Caveat: that would also suspend logging for any function called from my_func1 so be careful how you use it.

answered Sep 07 '11 at 22:21

Kirk Strauser

30,189
5
49
65

Good approach for a lot of applications. Thanks! – JS. Sep 08 '11 at 23:29
I guess this should have been decorator factory taking the logger to suspend. – Piotr Dobrogost Sep 19 '11 at 20:16
1

Please correct me if I am wrong, but looks to me that this would only disable one logger named `log`. If there are several loggers being used at the same time, e.g., in a Django application, how can this disable all loggers? – Subhamoy S. Nov 13 '13 at 13:53

score 3 · Answer 3 · answered Sep 07 '11 at 22:19

3

You could use a decorator:

import logging
import functools

def disable_logging(func):
    @functools.wraps(func)
    def wrapper(*args,**kwargs):
        logging.disable(logging.DEBUG)
        result = func(*args,**kwargs)
        logging.disable(logging.NOTSET)
        return result
    return wrapper

@disable_logging
def my_func1(...):

answered Sep 07 '11 at 22:19

unutbu

842,883
184
1,785
1,677

2

Hah! You beat me by seconds. Be careful inside `wrapper` as it won't re-enable logging of the `func()` call throws an exception. That's why I use the `try/finally` phrase to guarantee that logging is re-enabled regardless of whether `func()` succeeds. – Kirk Strauser Sep 07 '11 at 22:22
@KirkStrauser you should edit to add your solution. Your suggestions would be a great add on – sudeepgupta90 Jun 18 '20 at 17:55
This is the best answer. Here is a modified version that includes the try/finally block mentioned by @KirkStrauser. KirkStrauser's answer above has the try/finally but is missing the actual disable: ```from functools import wraps import logging def suspend_logging(func): @wraps(func) def inner(*args, **kwargs): logging.disable(logging.FATAL) try: return func(*args, **kwargs) finally: logging.disable(logging.NOTSET) return inner ``` – Scott Smith Mar 27 '22 at 22:46

score 1 · Answer 4 · answered Feb 24 '16 at 18:47

I took me some time to learn how to implement the sub-loggers as suggested by S.Lott.

Given how tough it was to figure out how to setup logging when I was starting out, I figure it's long past time to share what I learned since then.

Please keep in mind that this isn't the only way to set up loggers/sub-loggers, nor is it the best. It's just the way I use to get the job done to fit my needs. I hope this is helpful to someone. Please feel free to comment/share/critique.

Let's assume we have a simple library we like to use. From the main program, we'd like to be able to control the logging messages we get from the library. Of course we're considerate library creators, so we configure our library to make this easy.

First, the main program:

# some_prog.py

import os
import sys

# Be sure to give Vinay Sajip thanks for his creation of the logging module
# and tireless efforts to answer our dumb questions about it.  Thanks Vinay!
import logging

# This module will make understanding how Python logging works so much easier.
# Also great for debugging why your logging setup isn't working.
# Be sure to give it's creator Brandon Rhodes some love.  Thanks Brandon!
import logging_tree

# Example library
import some_lib

# Directory, name of current module
current_path, modulename = os.path.split(os.path.abspath(__file__))
modulename = modulename.split('.')[0]   # Drop the '.py'


# Set up a module-local logger
# In this case, the logger will be named 'some_prog'
log = logging.getLogger(modulename)

# Add a Handler.  The Handler tells the logger *where* to send the logging
# messages.  We'll set up a simple handler that send the log messages
# to standard output (stdout)
stdout_handler = logging.StreamHandler(stream=sys.stdout)
log.addHandler(stdout_handler)


def some_local_func():
    log.info("Info: some_local_func()")
    log.debug("Debug: some_local_func()")


if __name__ == "__main__":

    # Our main program, here's where we tie together/enable the logging infra
    # we've added everywhere else.

    # Use logging_tree.printout() to see what the default log levels
    # are on our loggers.  Make logging_tree.printout() calls at any place in
    # the code to see how the loggers are configured at any time.
    #
    # logging_tree.printout()

    print("# Logging level set to default (i.e. 'WARNING').")
    some_local_func()
    some_lib.some_lib_func()
    some_lib.some_special_func()

    # We know a reference to our local logger, so we can set/change its logging
    # level directly.  Let's set it to INFO:
    log.setLevel(logging.INFO)
    print("# Local logging set to 'INFO'.")
    some_local_func()
    some_lib.some_lib_func()
    some_lib.some_special_func()


    # Next, set the local logging level to DEBUG:
    log.setLevel(logging.DEBUG)
    print("# Local logging set to 'DEBUG'.")
    some_local_func()
    some_lib.some_lib_func()
    some_lib.some_special_func()


    # Set the library's logging level to DEBUG.  We don't necessarily
    # have a reference to the library's logger, but we can use
    # logging_tree.printout() to see the name and then call logging.getLogger()
    # to create a local reference.  Alternately, we could dig through the
    # library code.
    lib_logger_ref = logging.getLogger("some_lib")
    lib_logger_ref.setLevel(logging.DEBUG)

    # The library logger's default handler, NullHandler() won't output anything.
    # We'll need to add a handler so we can see the output -- in this case we'll
    # also send it to stdout.
    lib_log_handler = logging.StreamHandler(stream=sys.stdout)
    lib_logger_ref.addHandler(lib_log_handler)
    lib_logger_ref.setLevel(logging.DEBUG)

    print("# Logging level set to DEBUG in both local program and library.")
    some_local_func()
    some_lib.some_lib_func()
    some_lib.some_special_func()


    print("# ACK! Setting the library's logging level to DEBUG output")
    print("# all debug messages from the library.  (Use logging_tree.printout()")
    print("# To see why.)")
    print("# Let's change the library's logging level to INFO and")
    print("# only some_special_func()'s level to DEBUG so we only see")
    print("# debug message from some_special_func()")

    # Raise the logging level of the libary and lower the logging level
    # of 'some_special_func()' so we see only some_special_func()'s
    # debugging-level messages.
    # Since it is a sub-logger of the library's main logger, we don't need
    # to create another handler, it will use the handler that belongs
    # to the library's main logger.
    lib_logger_ref.setLevel(logging.INFO)
    special_func_sub_logger_ref = logging.getLogger('some_lib.some_special_func')
    special_func_sub_logger_ref.setLevel(logging.DEBUG)

    print("# Logging level set to DEBUG in local program, INFO in library and")
    print("# DEBUG in some_lib.some_special_func()")
    some_local_func()
    some_lib.some_lib_func()
    some_lib.some_special_func()

Next, our library:

# some_lib.py

import os
import logging

# Directory, name of current module
current_path, modulename = os.path.split(os.path.abspath(__file__))
modulename = modulename.split('.')[0]   # Drop the '.py'

# Set up a module-local logger.  In this case the logger will be
# named 'some_lib'
log = logging.getLogger(modulename)

# In libraries, always default to NullHandler so you don't get
# "No handler for X" messages.
# Let your library callers set up handlers and set logging levels
# in their main program so the main program can decide what level
# of messages they want to see from your library.
log.addHandler(logging.NullHandler())

def some_lib_func():
    log.info("Info: some_lib.some_lib_func()")
    log.debug("Debug: some_lib.some_lib_func()")

def some_special_func():
    """
    This func is special (not really).  It just has a function/method-local
    logger in addition to the library/module-level logger.
    This allows us to create/control logging messages down to the
    function/method level.

    """
    # Our function/method-local logger
    func_log = logging.getLogger('%s.some_special_func' % modulename)

    # Using the module-level logger
    log.info("Info: some_special_func()")

    # Using the function/method-level logger, which can be controlled separately
    # from both the library-level logger and the main program's logger.
    func_log.debug("Debug: some_special_func(): This message can be controlled at the function/method level.")

Now let's run the program, along with the commentary track:

# Logging level set to default (i.e. 'WARNING').

Notice there's no output at the default level since we haven't generated any WARNING-level messages.

# Local logging set to 'INFO'.
Info: some_local_func()

The library's handlers default to NullHandler(), so we see only output from the main program. This is good.

# Local logging set to 'DEBUG'.
Info: some_local_func()
Debug: some_local_func()

Main program logger set to DEBUG. We still see no output from the library. This is good.

# Logging level set to DEBUG in both local program and library.
Info: some_local_func()
Debug: some_local_func()
Info: some_lib.some_lib_func()
Debug: some_lib.some_lib_func()
Info: some_special_func()
Debug: some_special_func(): This message can be controlled at the function/method level.

Oops.

# ACK! Setting the library's logging level to DEBUG output
# all debug messages from the library.  (Use logging_tree.printout()
# To see why.)
# Let's change the library's logging level to INFO and
# only some_special_func()'s level to DEBUG so we only see
# debug message from some_special_func()
# Logging level set to DEBUG in local program, INFO in library and
# DEBUG in some_lib.some_special_func()
Info: some_local_func()
Debug: some_local_func()
Info: some_lib.some_lib_func()
Info: some_special_func()
Debug: some_special_func(): This message can be controlled at the function/method level.

It's also possible to get only debug messages only from some_special_func(). Use logging_tree.printout() to figure out which logging levels to tweak to make that happen!

Probably easier to use `__name__` to get your module name. https://stackoverflow.com/a/35326281/3427777 — fantabolous, Oct 24 '19 at 10:41

score 1 · Answer 5 · answered Mar 27 '22 at 22:50

This combines @KirkStrauser's answer with @unutbu's. Kirk's has try/finally but doesn't disable, and unutbu's disables w/o try/finally. Just putting it here for posterity:

from functools import wraps
import logging

def suspend_logging(func):
    @wraps(func)
    def inner(*args, **kwargs):
        logging.disable(logging.FATAL)
        try:
            return func(*args, **kwargs)
        finally:
            logging.disable(logging.NOTSET)
    return inner

Example usage:

from logging import getLogger
logger = getLogger()

@suspend_logging
def example()
  logger.info("inside the function")

logger.info("before")
example()
logger.info("after")

score 0 · Answer 6 · answered Aug 20 '20 at 11:40

If you are using logbook silencing logs is very simple. Just use a NullHandler - https://logbook.readthedocs.io/en/stable/api/handlers.html

>>> logger.warn('TEST')
[12:28:17.298198]  WARNING: TEST

>>> from logbook import NullHandler
>>> with NullHandler():
...    logger.warn('TEST')

Disable logging per method/function?

6 Answers6