0

The python logging module has a common pattern (ex1, ex2) where in each module you get a new logger object for each python module.

I'm not a fan of blindly following patterns and so I would like to understand a little bit more.

Why get a new logger object in each new module?

Why not have everyone just use the same root logger and configure the formatter with %(module)s?

Is there examples where this pattern is NECESSARY/NEEDED (i.e. because of some sort of performance reason[1])?

[1] In a multi-threaded python program, is there some sort of hidden synchronization issues that is fixed by using multiple logging objects?

Community
  • 1
  • 1
Trevor Boyd Smith
  • 18,164
  • 32
  • 127
  • 177
  • related question but probably needs a new stackoverflow question for itself: if you have multiple logging objects all using the same console, stream, and/or file handler is there a hidden synchronization issue (only one thread can read/write to the file at a time)? – Trevor Boyd Smith Sep 22 '16 at 15:31
  • `logging` is defined to be thread safe so loggers *do* use locks for writing their logs. This means that it is safe to use in a multithreaded environment. Note however that it is a well-known problem when using multi-processing on unix systems instead: by default `fork`ing doesn't copy locks and can result in deadlocks in rare conditions. See [here](http://stackoverflow.com/questions/24509650/deadlock-with-logging-multiprocess-multithread-python-script). – Bakuriu Sep 22 '16 at 15:52

2 Answers2

1

Each logger can be configured separately. Generally, a module logger is not configured at all in the module itself. You create a distinct logger and use it to log messages of varying levels of detail. Whoever uses the logger decides what level of messages to see, where to send those messages, and even how to display them. They may want everything (DEBUG and up) from one module logged to a file, while another module they may only care if a serious error occurs (in which case they want it e-mailed directly to them). If every module used the same (root) logger, you wouldn't have that kind of flexibility.

chepner
  • 497,756
  • 71
  • 530
  • 681
1

The logger name defines where (logically) in your application events occur. Hence, the recommended pattern

logger = logging.getLogger(__name__)

uses logger names which track the Python package hierarchy. This in turn allows whoever is configuring logging to turn verbosity up or down for specific loggers. If everything just used the root logger, one couldn't get fine grained control of verbosity, which is important when systems reach a certain size / complexity.

The logger names don't need to exactly track the package names - you could have multiple loggers in certain packages, for example. The main deciding factor is how much flexibility is needed (if you're writing an application) and perhaps also how much flexibility your users need (if you're writing a library).

Vinay Sajip
  • 95,872
  • 14
  • 179
  • 191
  • re "when systems reach a certain size / complexity": i guess in small/medium programs the pattern is not so important. but as your complexity/size grows... the pattern becomes more important/necessary. – Trevor Boyd Smith Sep 22 '16 at 15:49