logging to different files in different module

Question

I'm trying to write a file python modules and I need to log the output of different module to different files.

module_a.py

import logging
logging.basicConfig(levle = logging.INFO, filename="a.log")
logger = logging.getLogger(__name__)

def function_a():
    logger.info("this is function a")

if __name__ == "__main__":
    logger.info("this is module a")
    function_a()

module_b.py

import logging
import module_a

logging.basicConfig(levle = logging.INFO, filename="b.log")
logger = logging.getLogger(__name__)

def function_b():
    module_a.function_a()
    logger.info("this is function b")

if __name__ == "__main__":
    logger.info("this is module b")
    function_b()

What I want to do is, I wants module_b.py always log to b.log, and module_a.py always log to a.log, no matter the module was called or imported and then called.

The issue I got now is,

If I run

python3 module_a.py

All good.

If I run

python3 module_b.py

then all logs will be logged into a.log. I understand when import module_a, logger got overwritten. So how can make sure these two modules can always log into the right files?

Thanks

Have you tried the advice here? https://stackoverflow.com/questions/40495083/using-python-logging-from-multiple-modules-with-writing-to-a-file-and-rotatingfi — Andrew-Harelson, Oct 01 '21 at 04:33
@Andrew-Harelson I don't have a main file. So I could not define the handler in main.py. That's why make log configure a little bit difficult. — Free P, Oct 01 '21 at 04:42
2 things. When you run `python3 module_b.py`, then mobdule_b.py is your main file. There's no reason your main file has to be named main.py. Second, defining your handlers in your main file is just best practice, not a hard rule. If you have some kind of constraint where you can't modify the main file, you can always just setup your handlers in the module. — Andrew-Harelson, Oct 01 '21 at 04:51

score 0 · Answer 1 · answered Oct 01 '21 at 05:18

Normally, logging rules are decided by top level scripts. Imported modules just log without worrying about where the logs go. You want something different. So stay clear of basicConfig which is just a helper, and have each module setup its own logger.

In this example, I pulled the setup into a function. Its an easy way to get rid of temporary variables like handler. You may want to add a formatter while you are at it. The function is the same in both scripts. You may want to pull it into a single common utilities module. Or maybe there is a reason to do things differently in different modules. There are so many ways to setup loggers that the function is really just a rough guess of what you'll do in your project.

module_a

import logging

def make_logger(name):
    handler = logging.FileHandler(filename=name + ".log")
    logger = logging.getLogger(name)
    logger.addHandler(handler)
    logger.setLevel(logging.INFO)
    return logger

logger = make_logger("a")

def function_a():
    logger.info("this is function a")

if __name__ == "__main__":
    logger.info("this is module a")
    function_a()

module_b

import logging
import module_a

def make_logger(name):
    handler = logging.FileHandler(filename=name + ".log")
    logger = logging.getLogger(name)
    logger.addHandler(handler)
    logger.setLevel(logging.INFO)
    return logger

logger = make_logger("b")

def function_b():
    module_a.function_a()
    logger.info("this is function b")

if __name__ == "__main__":
    logger.info("this is module b")
    function_b()

logging to different files in different module

1 Answers1