8

I'm working on a small python project that has the following structure -

project 
 -- logs
 -- project
    __init.py__
    classA.py
    classB.py
    utils.py
 -- main.py

I've set up the logging configuration in __init.py__ under project as follows:

import logging
from logging import StreamHandler
from logging.handlers import RotatingFileHandler

# Create the Logger
logger = logging.getLogger(__name__)
logger.setLevel(logging.INFO)

# Create the Handler for logging data to a file
logger_handler = RotatingFileHandler('logs\\mylog.log', maxBytes=1024, backupCount=5)
logger_handler.setLevel(logging.INFO)

#Create the Handler for logging data to console.
console_handler = StreamHandler()
console_handler.setLevel(logging.INFO)

# Create a Formatter for formatting the log messages
logger_formatter = logging.Formatter('%(name)s - %(levelname)s - %(message)s')

# Add the Formatter to the Handler
logger_handler.setFormatter(logger_formatter)
console_handler.setFormatter(logger_formatter)

# Add the Handler to the Logger
logger.addHandler(logger_handler)
logger.addHandler(console_handler)

Setting up things this way seems to set up the root logger at the package level, and not at the entire project level. As a result, logged statements inside main.py don't appear in the log file, whereas all log statements in the two classes classA and classB as well as utils.py are routed to the console and the log file as expected.

How do I set up the logging so that I'm able to configure it once and use it across the project? I tried moving the logging configuration statements to main.py, but it didn't seem to work. 

def setupLogging():
    # Create the Logger
    logger = logging.getLogger(__name__)
    logger.setLevel(logging.INFO)
    ..etc..

def main():   
    setupLogging()

if __name__ == "__main__":
    main()
xuxu
  • 418
  • 5
  • 15

1 Answers1

10

It is correct to configure logging only once for the whole project, as you first tried, not each package separately.

What you did wrong is you configured the logger for the current module only:

logger = logging.getLogger(__name__)

Instead of that, you want to configure the root logger:

root_logger = logging.getLogger()
root_logger.setLevel(logging.INFO)

# ...

root_logger.addHandler(logger_handler)
root_logger.addHandler(console_handler)

The configuration done for the root logger applies to every logger which does not explicitly override it.

Then you should use the specific logger when actually logging:

logger = logging.getLogger(__name__)

logger.warning("I am warning you about %s", something)
zvone
  • 18,045
  • 3
  • 49
  • 77
  • Hi, thanks for the advise. It worked! I have a follow-up question though. Setting up the root logger this way seems to have set up the level for other modules imported and I'm now getting messages from third-party imported modules in the log file as well. How do I log messages only from the modules that I have written? – xuxu Dec 02 '17 at 13:18
  • You can set the level for each of them (right here where you are doing this), e.g. `logging.getLogger("product").setLevel(logging.INFO)` or `logging.getLogger("numpy").setLevel(logging.DEBUG)`. It may be easier to use logging configuration file, though (https://docs.python.org/2/library/logging.config.html#logging.config.fileConfig) – zvone Dec 02 '17 at 20:58
  • Im also having difficulty of configuring the logger. When I rename the root_logger ie `root_logger = logging.getLogger('some-name')`, other loggers in other modules don't work anymore. Do you know why? @zvone – addicted Dec 29 '19 at 06:29
  • 1
    @addicted You cannot "rename the root_logger". The root logger has no name. You access it with `logging.getLogger()`. What you did is you configured some other logger and not the root logger. – zvone Dec 29 '19 at 09:46
  • @zvone I want the other loggers to follow a certain application name as their prefix. Can I configure that logger and have the other modules' logger as a child to that configured logger? I want the child logger in other modules to have the main logger's name as a prefix in the logging name. – addicted Dec 29 '19 at 09:49
  • @addicted You can name any logger any way you want. You can also configure any logger any way you want. The point of having a configuration for the root logger is that it applies to all loggers, so what you do on it, you can do just once. In other words, you should usually *configure* the root logger and *log* to other loggers. – zvone Dec 29 '19 at 09:52
  • @zvone the reason I want variable naming in my other child loggers is because I will have 2 main modules that calls a common submodules. So I want this submodules to log with a name adhering to whatever main modules call it. Which is why I want to have the application name prefix in the logger name, it is so I can identify which of the 2 main modules call it. Example: `main1.submodule` for 1st main module and `main2.submodule` for 2nd main module. – addicted Dec 29 '19 at 09:57