Python difficulty understanding getLogger(__name__)

Question

Im quite confused on the logging docs' explanation of getLogger(__name__) as a best practice.

Gonna be explaining my entire thought process, feel free to leave a comment any time I make a mistake

The logging docs says

A good convention to use when naming loggers is to use a module-level logger, in each module which uses logging, named as follows: logger = logging.getLogger(__name__)

Say I have a project structure:

main_module.py
cookbook_example
---auxiliary_module.py

main_module.py

import logging
from cookbook_example import auxiliary_module
# Creates a new logger instance
logger = logging.getLogger(__name__)
logger.setLevel(logging.DEBUG)
# Create file handler that logs debug messages
fh = logging.FileHandler('spam.log', mode='w')
fh.setLevel(logging.DEBUG)
# Create a formatter
formatter = logging.Formatter(
    '%(asctime)s - %(name)s - %(levelname)s - %(message)s')
fh.setFormatter(formatter)
logger.addHandler(fh)

logger.info('Creating instance of auxiliary_module.Auxiliary')
a = auxiliary_module.Auxiliary()
logger.info('Calling auxiliary_module.do_something')
a.do_something()
auxiliary_module.some_function()

auxiliary_module.py

import logging

# create logger
module_logger = logging.getLogger(f'__main__.{__name__}')


def some_function():
    module_logger.info('received a call to "some_function"')

Now, from this SO thread, I infer that getLogger(__name__) should not actually be used in EVERY module that uses logging but instead in the module where a logger is configured, which in this case would be main_module.py

e.g. In the auxiliary module, trying to get the custom logger through getLogger(__name__) will return the root logger, whereas getLogger(f'__main__.{__name__}') will return that custom logger.

To me, this formatting of getLogger(f'__main__.{__name__}') doesn't seem much easier to write than the explicit getLogger('main_module.auxiliary_module'). Furthermore, in the log files it logs __main__.auxiliary_module rather than main_module.auxiliary_module, losing a bit of accuracy.

Lastly, I previously stated that to my understanding, getLogger(__name__) should only be placed in the module where the logger is configured. However, configuration should be placed in a config file or dict anyways.

Thus, I don't seem to understand any reasonable usage of getLogger(__name__) and how it is, according to the docs, a best practice. Could someone explain this and maybe link a repo that uses loggers with proper organisation that I could refer to? Thanks

Answer 1

Assume this simple project:

project/
├── app.py
├── core
│   ├── engine.py
│   └── __init__.py
├── __init__.py
└── utils
    ├── db.py
    └── __init__.py

Where app.py is:

import logging
import sys

from utils import db
from core import engine

logger = logging.getLogger()
logger.setLevel(logging.INFO)
stdout = logging.StreamHandler(sys.stdout)
stdout.setFormatter(logging.Formatter("%(name)s: %(message)s"))
logger.addHandler(stdout)


def run():
    db.start()
    engine.start()


run()

and utils/db.py and core/engine.py is:

from logging import getLogger

print(__name__)  # will print utils.db or core.engine
logger = getLogger(__name__)
print(id(logger))  # different object for each module; child of root though


def connect():
    logger.info("started")

If you run this using python app.py, you will see that it takes care of printing the proper namespaces for you.

utils.db: started
core.engine: started

If your code is well organised, your module name itself is the best logger name available. If you had to reinvent those names, it usually means that you have a bad module structure (or some special, non-standard use case). For most purposes though, this should work fine (hence part of stdlib).

That is all there is to it. Remember, you don't really set handlers for libraries; that is left to the consumer.