32

I am currently developing a package which can be used without writing any new code and the modules can be used to develop new code (see documentation).

Many of my modules use logging in a very simple way:

import logging
import sys
logging.basicConfig(format='%(asctime)s %(levelname)s %(message)s',
                    level=logging.DEBUG,
                    stream=sys.stdout)
# Code of the module follows

I have these lines in many modules. It seems to me that I should factor it out, but I am not sure what the best / recommended / most pythonic way to do so is.

The relevant parts in the logging documentation seem to be Configuring Logging for a Library and Logging from multiple modules.

I guess I should simply move the line logging.basicConfig to the executables (bin/hwrt) and remove all other logging.basicConfig lines.

Is there any rule how packages should use logging (like PEP8 for coding style)?

If other developers use my code they might want to disable / modify the way my package does logging (so that it doesn't get mixed with their logging calls). Is there a way to help them do so?

Martin Thoma
  • 124,992
  • 159
  • 614
  • 958
  • To my opinion, the solution is beter explained on [this stackoverflow post](https://stackoverflow.com/questions/15727420/using-logging-in-multiple-modules). – Siete Aug 19 '22 at 11:28

1 Answers1

50

Your library should not configure logging; that's an application-wide task.

Instead, just set up your logger object based on __name__, use that, and that's it. Document that you use logging, and developers using your library can use the standard logging API to configure logging.

You could add a null handler to your root logger (the logger registered for your package name) to prevent a default configuration being used if the application didn't set one:

# This goes into your library somewhere
logging.getLogger('name.of.library').addHandler(logging.NullHandler())

and developers using your library can then disable all logging just for your library by disabling log propagation:

logging.getLogger('name.of.library').propagate = False

All this is already documented in the logging module; you can consider it the style guide for Python logging. From the Configuring Logging for a Library section you already linked to:

Note: It is strongly advised that you do not add any handlers other than NullHandler to your library’s loggers. This is because the configuration of handlers is the prerogative of the application developer who uses your library. The application developer knows their target audience and what handlers are most appropriate for their application: if you add handlers ‘under the hood’, you might well interfere with their ability to carry out unit tests and deliver logs which suit their requirements.

logging.basicConfig() does just that; it creates handler configuration.

Martijn Pieters
  • 1,048,767
  • 296
  • 4,058
  • 3,343
  • "Instead, just set up your logger object based on __name__," What does that mean? – Martin Thoma Nov 21 '14 at 06:26
  • 1
    "`logging.basicConfig()` does just that; it creates handler configuration" - Just to make sure: This means I should not use `logging.basicConfig()` in my library code, but in the "executable" part I can use it, can't I? – Martin Thoma Nov 21 '14 at 06:27
  • 1
    @moose: yes, absolutely. Your 'executable' part is an application. The `__name__` global in a module is the module name, including parent packages. `logging` creates a hierarchy of logging objects based on `.` delimiters, so a logger in `foo.bar` propagates up to `foo`; by using `logger = logging.getLogger(__name__)` in each module in your library you make it easier on yourself in that respect. – Martijn Pieters Nov 21 '14 at 09:08
  • Within your package, you are adding the null handler to the logger which has name as 'name.of.library'. What about the root logger within the package? I mean - once the 'name.of.library' logger handles the log (using nullhandler), and then propagates it to the root logger, will the package's root logger not print to command line (since root logger uses the lastResort handler if not configured)? – variable Oct 21 '19 at 14:05
  • @variable The logged in my example using `nullhandler` *is the library root logger*. – Martijn Pieters Oct 21 '19 at 18:30
  • @variable: would it be clearer if I gave a sample name for a hypothetical library, like *acme* and then use that instead of `name.of.library`? – Martijn Pieters Oct 21 '19 at 18:32
  • 1
    @variable last but not least: using a nullhandler in your library also prevents log messages coming from your library being routed to the stderr fallback when the root logger is not configured. That’s the whole point of using one. – Martijn Pieters Oct 21 '19 at 18:36
  • @MartijnPieters Is there a simple example repository somewhere that you can share? It is still confusing on how python developers use do this. I place my logging configuration in my library's `__init__.py` module and import it in other modules. I'm more concerned about where to place the logging configuration -- once. – pyeR_biz Oct 22 '21 at 06:01
  • 4
    @pyeR_biz: if you are building a library you **don't create logging configuration**. Set the configuration in the *application* that uses the library. You could follow the `urllib3` example, they follow the [documented practice for libraries](https://docs.python.org/3/howto/logging.html#library-config) and [set a Null handler](https://github.com/urllib3/urllib3/blob/c5bc7163022c0ac2e765cdb7309937a53e32d368/src/urllib3/__init__.py#L45). They also have a [utility to aid debugging](https://github.com/urllib3/urllib3/blob/c5bc7163022c0ac2e765cdb7309937a53e32d368/src/urllib3/__init__.py#L48-L63). – Martijn Pieters Oct 29 '21 at 11:59
  • 3
    @pyeR_biz: compare that to a typical application such as MusicBrainz Picard, which [configures logging](https://github.com/metabrainz/picard/blob/3ea43252c498bd27aa16ad5b7899c2b55c1027b5/picard/log.py). – Martijn Pieters Oct 29 '21 at 12:01