3

I am heavily using python threading, and my many use-cases require that I would log separate task executions under different logger names.

A typical code example would be:

def task(logger=logger):
    global_logger.info('Task executing')
    for item in [subtask(x) for x in range(100000)]:
        # While debugging, one would generally prefer to see these
        global_logger.debug('Task executed item {}'.format(item))
    global_logger.info('Task done')

def thread_task(task_index, logger=logger):
    task(logger=global_logger.getChild('main.task.{}'.format(task_index)))

def main():
    pool = Pool(10)
    for item in pool.map(thread_task, range(128))
        pass

Is there any standard approach in python logging module that would allow me to set the logging level for all threads at once?

In code, what I am trying to do is:

# For all threads
logger.getChild('main.task.0').setLevel('INFO')
logger.getChild('main.task.1').setLevel('INFO')
logger.getChild('main.task.2').setLevel('INFO')

I should note that am aware that I could set the levels inside task_thread. My question is there to find out if this can be done easier.

Andrey Cizov
  • 695
  • 10
  • 20

2 Answers2

3

Because loggers inherit their parent's level if not explicitly set, you could just do e.g.

root_name = global_logger.name
logging.getLogger(root_name + '.main.task').setLevel(logging.INFO)

and that would mean that all child loggers inherit that level, unless a level were explicitly set for one of them.

Note that unless you want to attach different handlers to the different thread loggers, you don't get much benefit from having a logger per thread - you can always use other methods to put the task_index value in the log.

Vinay Sajip
  • 95,872
  • 14
  • 179
  • 191
  • You're missing the point. It's an easy solution, but is still a hack. What if I would like to do: `logging.getLogger('main.task.*.subtask_a').setLevel(logging.INFO)` `logging.getLogger('main.task.*.subtask_b').setLevel(logging.DEBUG)` – Andrey Cizov Jan 24 '16 at 17:23
  • I have now seen your edit. That makes much more sense. – Andrey Cizov Jan 24 '16 at 17:32
1

I was having the same issue suppressing the output from the BAC0 library. I tried changing the parent logger and that did not update the children (Python 3.9)

This answer is based on the accepted answer in this post: How to list all existing loggers using python.logging module

for name in logging.root.manager.loggerDict:
    if name.startswith("main.task."):
        logging.getLogger(name).setLevel(logging.INFO)

You could probably replace the name.startswith("main.task.") with some regex matching for wild card notation, but this should do the trick for you.

What it does is it gets and iterates over every logger which has been configured, and checks if the name matches your search criteria. If it does, it will get the logger by that name, and set the level to INFO.

CPSuperstore
  • 633
  • 10
  • 18