python - manage different packages logger

Question

I have read :

Using logging in multiple modules,

Managing loggers with Python logging

https://docs.python.org/3/howto/logging-cookbook.html

and other posts.

and still, I don't get it (something is really wrong in the way the logging module works IMO).

Let's see a straight forward example:

module1:

import logging
logger=logging.getLogger(__name__)

def test_log():
    logger.info("module1")
    logger.debug("module1")

module2:

import logging
logger=logging.getLogger(__name__)

def test_log2():
    logger.info("module2")
    logger.debug("module2")

main.py

import module1
import module2
import logging

# create logger
logger = logging.getLogger(__name__)
logger.setLevel(logging.DEBUG)
# create console handler with a higher log level
ch = logging.StreamHandler()
ch.setLevel(logging.DEBUG)
# create formatter and add it to the handlers
formatter = logging.Formatter('%(asctime)s - %(name)s - %(levelname)s - %(message)s')
ch.setFormatter(formatter)
# add the handlers to the logger
logger.addHandler(ch)

logger.info("main debug")
logger.info("main info")
module1.test_log()
module2.test_log2()

This outputs:

2022-10-28 11:18:09,101 - __main__ - INFO - main debug
2022-10-28 11:18:09,102 - __main__ - INFO - main info

The 2 others modules loggers did not work ...

Also, changing logger.setLevel(logging.DEBUG) to logger.setLevel(logging.ERROR) even if there is ch.setLevel(logging.DEBUG) remove the logging below ERROR level.

But the opposite is ALSO TRUE ! Setting ch.setLevel(logging.DEBUG) to ch.setLevel(logging.ERROR) even if there is logger.setLevel(logging.DEBUG) remove the logging below ERROR level.

What's the point of setting two differents things if they must be equal to work anyway ?

But ! with :

logger.setLevel(logging.DEBUG)
ch.setLevel(logging.INFO)

or with

logger.setLevel(logging.INFO)
ch.setLevel(logging.DEBUG)

we get both messages. I can't make sense of how this is supposed to work.

Anyway, what about my 2 modules ?

logging.getLogger("module1").setLevel(logging.DEBUG)
logging.getLogger("module2").setLevel(logging.DEBUG)

does not fix anything

Finally !

logging.getLogger("module1").setLevel(logging.DEBUG)
logging.getLogger("module2").setLevel(logging.INFO)
logging.basicConfig(level=logging.DEBUG)

logger.info("main debug")
logger.info("main info")
module1.test_log()
module2.test_log2()

outputs:

2022-10-28 11:27:47,843 - __main__ - INFO - main debug
2022-10-28 11:27:47,843 - __main__ - INFO - main debug
2022-10-28 11:27:47,844 - __main__ - INFO - main info
2022-10-28 11:27:47,844 - __main__ - INFO - main info
2022-10-28 11:27:47,844 - module1 - DEBUG - module1
2022-10-28 11:27:47,845 - module1 - INFO - module1
2022-10-28 11:27:47,845 - module2 - INFO - module2

which is a failure because,

main logging has been written twice
it seems really innapropriate to me to go through basicConfig, especially considering that the users of my package should be bothered to deal with the formatting and other options of logging (which in my case seems necessary)
if you want debug level on one module you need to specify DEBUG to basicConfig which result every other package to print in DEBUG level, then forcing the user to set EVERY library levels manually.

So, how should we implement the logger when we have differents modules, packages et different level of logging for each one of them ?

Thanks you !

python - manage different packages logger

0 Answers0