Redirecting Loggers`messages to sys.stdout and sys.stderr

Question

I´m just starting to learn Python and have encountered a Problem that I´m not able to solve. I want to redirect every level above CRITICAL to sys.stderr and everything above WARNING to sys.stdout. I came up with this script...

import logging
import sys


print("imported module {}".format(__name__))


class PyLogger(logging.Logger):
    """Wrapper for logging.Logger to redirect its message to
    sys.stdout or sys.stderr accordingly """

    def __init__(self, *args):
        super(PyLogger, self).__init__(self, *args)

        # get Logger
        logger = logging.getLogger()
        logger.setLevel(logging.DEBUG)

        # build Formatter
        formatter = logging.Formatter(fmt="%(asctime)s:%(name)s   %(message)s")

        # build StreamHandler for sys.stderr
        error = logging.StreamHandler(stream=sys.stderr)
        error.setLevel(logging.CRITICAL)
        error.setFormatter(formatter)
        logger.addHandler(error)

        # build StreamHandler for sys.stdin
        out = logging.StreamHandler(stream=sys.stdout)
        out.setFormatter(formatter)
        out.setLevel(logging.WARNING)
        logger.addHandler(out)


def main():
    logger = PyLogger()
    # help(logger)
    logger.info("INFO")


if __name__ == "__main__":
    main()

When running this scrip directly I get the following error:

No handlers could be found for logger "<__main__.PyLogger object at 0x105f23c50>"

I´ve googled around and many people said that a logging.basicConfig() would do the job but that didn´t worked for me.

Maybe someone of you guys could help me out. Thanks!

You can try just single clicking at the beginning of the code, and then shift clicking at the end, which will highlight everything. — sorak, Mar 01 '18 at 17:11

ely · Accepted Answer · 2018-03-01T17:09:02.260

Your class subclasses logging.Logger, so you should not call getLogger or manipulate a logger as an attribute. Rather, the logger is self inside the class, and should be adjusted directly:

import logging
import sys


print("imported module {}".format(__name__))


class PyLogger(logging.Logger):
    """Wrapper for logging.Logger to redirect its message to                                                                                   
    sys.stdout or sys.stderr accordingly """

    def __init__(self, *args):
        super(PyLogger, self).__init__(self, *args)

        #####
        # self *is* the logger!                                                                                                                          
        self.setLevel(logging.DEBUG)

        # build Formatter                                                                                                                      
        formatter = logging.Formatter(fmt="%(asctime)s:%(name)s   %(message)s")

        # build StreamHandler for sys.stderr                                                                                                   
        error = logging.StreamHandler(stream=sys.stderr)
        error.setLevel(logging.CRITICAL)
        error.setFormatter(formatter)

        #####
        # Assign the handler to self
        self.addHandler(error)

        # build StreamHandler for sys.stdin                                                                                                    
        out = logging.StreamHandler(stream=sys.stdout)
        out.setFormatter(formatter)
        out.setLevel(logging.WARNING)

        #####
        # Assign the handler to self
        self.addHandler(out)


def main():
    logger = PyLogger()
    # help(logger)                                                                                                                             
    logger.info("INFO")
    logger.warning("WARN")
    logger.critical("CRIT")


if __name__ == "__main__":
    main()

This displays the following, as expected:

ely@eschaton:~/programming$ python test_logger.py 
imported module __main__
2018-03-01 11:59:41,896:<__main__.PyLogger object at 0x7fa236aa4a50>   WARN
2018-03-01 11:59:41,896:<__main__.PyLogger object at 0x7fa236aa4a50>   CRIT
2018-03-01 11:59:41,896:<__main__.PyLogger object at 0x7fa236aa4a50>   CRIT

Notice how the critical message trips two different output handlers, so it appears twice (once because it satisfied warning level, once for critical level).

In your original code, notice that you are creating a variable called logger inside of __init__, but this not assigned to self or anything. This variable gets destroyed when it goes out of scope of the __init__ function, and so the assignment of any handlers is meaningless. Plus, because handlers weren't assigned to self, but the object that self is referencing is the logger that will be called later on, that is why you see the error about no handlers.

Ok I see! Thank you :) So basically if i inherit from any class... by calling super(class, self).__init__() i will always get an instance of that class. So no need for any "self.getAnything()" because I´ve already got that instance?! And if I now want to get rid of the double logging I would need to specify a Filter for one of the StreamHandlers correct? — tweak-wtf, Mar 02 '18 at 14:17
@TonyDorfmeister If you want to access methods from the `Logger` class like `addHandler` or `setLevel`, then you need to do it with `self`, because `self` represents the instance of `PyLogger` (similar to `this` in C++ or Java), and `PyLogger` inherits those methods of `Logger`. But you would not need to create a logger with `logging.getLogger`. — ely, Mar 02 '18 at 15:51
You are correct that you can use `addFilter` with a custom filter in order to customize how you handle levels in each logger. [Here is an example](https://stackoverflow.com/a/8163115/567620) which I believe would work directly for your use case. — ely, Mar 02 '18 at 15:53
Surely you wouldn't want the same log logged twice. EG The critical log should only go to stderr. — Jonathan Wylie, Jun 20 '19 at 16:09
@JonathanWylie no, you are incorrect. In this example, there are two handlers set up. One reports anything of level warning or higher, the other outputs anything of level critical or higher, and they operate independently of each other. That can lead to multiple logging, but that is intended in this case. If you need different behavior, you would probably choose to use logging [Filters](http://docs.python.org/library/logging.html#filter-objects) to customize the business logic of not double-logging in one handler or the other. — ely, Jun 25 '19 at 18:55
@ely I understand what the handlers are doing and that they are therefore logging it twice. I suppose it depends on how you interact with the script. If you were to run such a script manually you would see the same log twice which is a bit rubbish. If you are redirecting the output streams to different places, then I can imagine it being usefull. Reading the question again it is doing what they asked. — Jonathan Wylie, Oct 08 '19 at 10:36

Redirecting Loggers`messages to sys.stdout and sys.stderr

1 Answers1