Customize logging for external/third-party libs

Question

I followed the advice of the django docs, and use logging like this:

import logging
logger = logging.getLogger(__name__)

def today(...):
    logger.info('Sun is shining, the weather is sweet')

With my current configuration, the output looks like this:

2016-08-11 14:54:06 mylib.foo.today: INFO Sun is shining, the weather is sweet

Unfortunately some libraries which I can't modify use logging like this:

import logging

def third_party(...):
    logging.info('Make you want to move your dancing feet')

The output unfortunately looks like this:

2016-08-09 08:28:04 root.third_party: INFO Make you want to move your dancing feet

I want to see this:

2016-08-09 08:28:04 other_lib.some_file.third_party: INFO Make you want to move your dancing feet

Difference:

root.third_party ==> other_lib.some_file.third_party

I want to see the long version (not root) if code uses logging.info() instead of logger.info()

Update

This is not a duplicate of Elegant setup of Python logging in Django, since the solution of it is:

Start of quote

In each module, I define a logger using

logger = logging.getLogger(__name__)

End of quote.

No, I won't modify third-party-code which uses logging.info() instead of logger.info().

Follow Up Question

Avoid logger=logging.getLogger(__name__) without loosing way to filter logs

https://docs.djangoproject.com/en/1.9/topics/logging/ you should configure it in your settings.py — be_good_do_good, Aug 11 '16 at 13:28
Possible duplicate of [Elegant setup of Python logging in Django](http://stackoverflow.com/questions/1598823/elegant-setup-of-python-logging-in-django) — be_good_do_good, Aug 11 '16 at 13:30
@be_good_do_good yes, I configured logging. The question is: How to handle loggers which are not named with `logger = logging.getLogger(__name__)` — guettli, Aug 11 '16 at 13:30
Also, I'd probably file a bug report with the upstream library. There's no excuse for *not* using `logger = logging.getLogger(__name__)` in a module. — Wayne Werner, Aug 11 '16 at 14:27
Could you partially solve this issues by logging file name? File name should respect PYTHONPATH, therefore should be unique. It is not logger name, but should be pretty close. — aisbaa, Aug 16 '16 at 10:31

score 4 · Accepted Answer · answered Aug 11 '16 at 14:22

As Wayne Werner suggested, I would use the Log Record format options. Here's an example.

File 1: external_module

import logging
def third_party():
    logging.basicConfig(level=logging.DEBUG)
    logger = logging.getLogger()

    logger.info("Hello from %s!"%__name__)

File 2: main

import external_module
import logging

logging.basicConfig(level=logging.DEBUG,
                    format='%(asctime)s %(module)s.%(funcName)s: %(levelname)s %(message)s')
logger = logging.getLogger(__name__)

def cmd():
    logger.info("Hello from %s!"%__name__)
    external_module.third_party()
cmd()

Output:

2016-08-11 09:18:17,993 main.cmd: INFO Hello from __main__!
2016-08-11 09:18:17,993 external_module.third_party(): INFO Hello from external_module!

score 3 · Answer 2 · answered Aug 11 '16 at 13:41

3

That's because they're using the root logger (which is what you get by default when you just do

import logging

logging.info("Hi! I'm the root logger!")

If you want to do something different you have two (or three) options. The best would be to use the Log Record format options. Alternatively, you could monkey patch the libraries that you're using, e.g.

import logging
import mod_with_lazy_logging

mod_with_lazy_logging.logger = logging.getLogger(mod_with_lazy_logging.__name__)

Or you could do something gnarly with parsing the ast and rewriting their bits of logging code. But, don't do that.

answered Aug 11 '16 at 13:41

Wayne Werner

49,299
29
200
290

Monkey Patching the library could work. But it has a lot of files ... It feels like too much work. – guettli Aug 15 '16 at 09:17
If it's an open source library, I'd recommend either filing an issue with the maintainer, or creating a PR yourself. Or both. Not using `logger = logging.getLogger(__name__)`, or at least using `'library name'` seems lazy/irresponsible. – Wayne Werner Aug 15 '16 at 14:28

Customize logging for external/third-party libs

2 Answers2