12

I have written a bunch of scripts over time and I am in the process of refatoring the scripts to keep the code DRY. I am currently using something along these lines in the various scripts:

if __name__ == '__main__':
    logger = logging.getLogger('dbinit')
    hdlr = logging.FileHandler('/var/logs/tmp/foo.log')
    formatter = logging.Formatter('%(asctime)s %(levelname)s %(message)s')
    hdlr.setFormatter(formatter)
    logger.addHandler(hdlr) 
    logger.setLevel(logging.WARNING)

Rather than repeating this in every script (i.e. "module"), I would like to have this logger initialisation done one somewhere and accessed by the various scripts (Hmm, maybe wrap in a singleton class?).

If I can't do that (i.e. put the logger initialisation code in one core module), I assume that by using the same log file name in the logging.FileHandler() call, the various scripts will write to the same file.

Is this assumption correct?

Last but not the least, what is the best practise (i.e. Pythonic) way to solve this problem?

Homunculus Reticulli
  • 65,167
  • 81
  • 216
  • 341

3 Answers3

2

create a function in your python module like below:

def createLogHandler(job_name,log_file):
    logger = logging.getLogger(job_name)
    ## create a file handler ##
    handler = logging.FileHandler(log_file)
    ## create a logging format ##
    formatter = logging.Formatter('%(asctime)s - %(name)s - %(levelname)s - %(message)s')
    handler.setFormatter(formatter)
    logger.addHandler(handler)
    return logger

Now call the function in you program like this:

job_name = 'dbinit'
log_file = '/var/logs/tmp/foo.log'
logger = createLogHandler(job_name ,log_file )
logger.info('Logger has been created')
Aravinth s.r
  • 21
  • 1
  • I don't think this will work as expected; you end up with multiple handlers referencing the same file and they will likely either trample on each other or have the potential for output to be interleaved. – dsz Aug 12 '23 at 03:27
2

Given that you are using if __name__ == __main__, I am assuming that these scripts would be running as different processes. If that's the case, then you should use a separate configuration file.

This configuration can be in stored in a file in the format as specified in the docs. Then, you can use logging.config.fileConfig to load the file. You can also have the configuration stored in JSON/YAML formats, convert it into a dictionary and load it using logging.config.dictConfig. The latter is the current recommended approach, although I find the former more straight forward. Read this for more information.

The advantages of using a config file approach are many fold:

  1. It keeps the configuration settings separated from your code
  2. It lets non-programmers make changes to configuration, as they are stored in easy to read formats
  3. It also keeps you from repeating yourself as you have already mentioned.
Phani
  • 3,267
  • 4
  • 25
  • 50
-1

(...) addHandler() will not add a handler if the handler already exist so having such init code in many places won't hurt.

If I can't do that (i.e. put the logger initialisation code in one core module)

You can to this and you should do this if you want to ensure initialization code is run once.

Community
  • 1
  • 1
Piotr Dobrogost
  • 41,292
  • 40
  • 236
  • 366
  • I think you are referencing [this answer](https://stackoverflow.com/a/6334064/39396) when you say "addHandler() will not add a handler if the handler already exist ". What you say is true only if it is the same handler object (`hdlr` in that answer.) What you say is not true if the handlers are created each time. (See [this answer](https://stackoverflow.com/a/31800084/39396) to the same question.) In this question, where each file may create a new handler (`logging.FileHandler(...)`) your answer is incorrect. – Carl G Nov 17 '17 at 13:48