Display name of original module when calling another module

Question

I'm creating centralised logging. That basically looks like the scripts below.

The logit module will create a file, based on the scripts name which called it. in this case apiCaller.

Originally i had this manually defined when calling logit, however i was searching for away for logit to determine the origin of the log itself.

There are 3 modules at play here:

main.py:

def runAnalytic(script):
    importlib.import_module("monitoringScripts."+script["package"]+"."+script["module"], package=None)

packageModule = [{"package":"awesome","module":"apiCaller"}]

with concurrent.futures.ThreadPoolExecutor() as executor:
    results = executor.map(runAnalytic, packageModule)

apiCaller.py (module above)

from adminTools.logger import logit
logit.create(results[i]["items"][r]["userId"],"apiCaller") #How i currently pass the script name, i want to get rid of this.

logit.py Handles all log requires from all my other scripts (centralised logging)

import sys, logging, logging.handlers, pathlib
#Path for all log files for scriptHub
logdir = str(pathlib.Path(__file__).parent.absolute())

#Creates the log file based on a given name from the script
def create(logMessage,scriptName, level="DEBUG"):
    #create filename
    log_filename = logdir+"/sysLogs/"+scriptName+".logs"
    #Creates the logging object
    my_logger = logging.getLogger(scriptName)
    my_logger.setLevel(logging.DEBUG)
    #Formats the log:
    formatter = logging.Formatter('%(asctime)s - %(message)s - %(name)s')
    #Gives the log file a limit for 100mb if it goes bigger than this, it will create another file, but keep the old one
    handler = logging.handlers.RotatingFileHandler(log_filename, maxBytes=100000000, backupCount=1)
    handler.setFormatter(formatter)
    #Handlers need to be cleared to stop duplicated logs.
    if (my_logger.hasHandlers()):
        my_logger.handlers.clear()
    my_logger.addHandler(handler)
    #creates the log message
    my_logger.debug(logMessage)

So, I'm not sure if that helps or hinders you all lol

Essentially, instead of providing logit with the script name, i want logit to get it from the module it's called from. E.g in this case "apiCaller" would be the name that's passed through to logit.

Answer 1

The question isn't very clear, but you can use inspect.stack().

loggy.py

import inspect

def log(s):
    caller = inspect.stack()[1]
    print(f"{caller.filename} line {caller.lineno} says: {s}")

thing.py

import loggy

loggy.log("Hey!")

---

/v/f/n/8/T/tmp.ZiRzgsqi $ python3 thing.py
thing.py line 3 says: Hey!
/v/f/n/8/T/tmp.ZiRzgsqi $

Answer 2

Okay, with the rewritten question:

I've seen it done the other way around than you do it - get a logger, then set it up (two lines in the module, not one). The logger is per module thing and always is there.

In your case, you re-get the logger and remake the handlers each time.

This way you can't make use of the beautiful possibilities logging module offers!

---

So basically, this other approach is:

In each script you do logger = logging.getLogger(__name__), usually somewhere near the top, below imports.

+You just call logit.setupLogger(logger). (In your case, in the next line. In case of scripts, I keep it in main function - so that if I ever import the script as a module, I will call whatever logging setup I need on imported_module.logger so it doesn't spam the wrong log file. :D)

Rewritten logit.py:

import sys, logging, logging.handlers, pathlib
#Path for all log files for scriptHub
logdir = str(pathlib.Path(__file__).parent.absolute())

#Creates the log file based on a given name from the script
def create(my_logger, level=logging.DEBUG):
    #create filename
    log_filename = logdir+"/sysLogs/"+logger.name+".logs"

    my_logger.setLevel(level)
    #Formats the log:
    formatter = logging.Formatter('%(asctime)s - %(message)s - %(name)s')
    #Gives the log file a limit for 100mb if it goes bigger than this, it will create another file, but keep the old one
    handler = logging.handlers.RotatingFileHandler(log_filename, maxBytes=100000000, backupCount=1)
    handler.setFormatter(formatter)
    #Handlers need to be cleared to stop duplicated logs.
    if (my_logger.hasHandlers()):
        my_logger.handlers.clear()
    my_logger.addHandler(handler)

This way, you only set up insides of the logger - including file handler - in the logit, and you can use standard logging things:

• you can use any level logging in your module:

logger.info("like")
logger.warning("this")

• you write every logging message in the code like above - make the code full of logging messages! - remember that debug messages should have everything needed for debug, this sometimes includes huge chunks of information + remember that debug messages may exist next to info messages, just with different details (e.g. "Getting info from X" is info, "Sent request 'some/address/here' and received '''chunk of data here'''" is debug).
• when you need to cut the level - e.g. you debugged your message and got tired of seeing so.much.info (or just going from dev to prod) - you just change logit.setupLogger(logger) to logit.setupLogger(logger, logging.INFO) or whatever level you need.

---

Logging might seem like a good idea to do your way, but logging module is quite powerful when you learn how to use it. :) Here's a How-To from Python's docs, but it has a lot of info, so simpler tutorials on python logging are a better thing to start with.

Tbh I started with my own logging practices as well, even after reading the docs and tutorials because I didn't get it at all. I only switched to the approach above when I saw it used in a lib I'm using. :)