Python-threading: separate logging of individual threads

Question

I would like individual .log file for each thread. Unfortunately, after using logging.basicConfig, many different files are created for logs, but finally all logs end up in the last declared file. What should threads do to have independent log files?

import logging
import threading
import time
from datetime import datetime

def test_printing(name):
    logging.basicConfig(
        format="%(asctime)s, %(levelname)-8s | %(filename)-23s:%(lineno)-4s | %(threadName)15s: %(message)s",  # noqa
        datefmt="%Y-%m-%d:%H:%M:%S",
        level=logging.INFO,
        force=True,
        handlers=[
            logging.FileHandler(f"{name}.log"),
            logging.StreamHandler()])
    logging.info(f"Test {name}")
    time.sleep(20)
    logging.info(f"Test {name} after 20s")


def function_thread():
    timestamp = datetime.now().strftime("%Y%m%d_%H%M%S")
    thread = threading.Thread(
            target=test_printing,
            kwargs={"name": timestamp}
        )
    thread.start()

for i in range(5):
    time.sleep(1)
    function_thread()

This should help... https://stackoverflow.com/a/61877582/2836621 — Mark Setchell, Feb 23 '22 at 14:44
@MarkSetchell Can you help me implement the code from your link in my code? preferably as a "new answer" :) — Cierniostwor, Feb 23 '22 at 15:19

G. C. · Accepted Answer · 2022-02-23T16:32:51.817

From https://docs.python.org/3/library/logging.html#logger-objects

Note that Loggers should NEVER be instantiated directly, but always through the module-level function logging.getLogger(name).

So you have to create and configure a new logger inside each thread:

logger = logging.getLogger(name)
logger.basicConfig(...)

more info at: https://docs.python.org/3/howto/logging.html#logging-from-multiple-modules

Edit: Use already defined name as logger identifier, instead of __name__

Edit: You cannot use logging.basicConfig, instead you need to configure each thread logger on its own. Full code provided and tested:

import logging
import threading
import time
from datetime import datetime

def test_printing(name):
    logger = logging.getLogger(name)
    logger.setLevel(logging.INFO)
    formatter = logging.Formatter(
        fmt="%(asctime)s, %(levelname)-8s | %(filename)-23s:%(lineno)-4s | %(threadName)15s: %(message)s",
        datefmt="%Y-%m-%d:%H:%M:%S")
    sh = logging.StreamHandler()
    fh = logging.FileHandler(f"{name}.log")
    sh.setFormatter(formatter)
    fh.setFormatter(formatter)
    logger.addHandler(sh)
    logger.addHandler(fh)

    logger.info(f"Test {name}")
    time.sleep(20)
    logger.info(f"Test {name} after 20s")


def function_thread():
    timestamp = datetime.now().strftime("%Y%m%d_%H%M%S")
    thread = threading.Thread(
            target=test_printing,
            kwargs={"name": timestamp}
        )
    thread.start()


for i in range(5):
    time.sleep(1)
    function_thread()

Python-threading: separate logging of individual threads

1 Answers1