Python Logging different file on each turn

Question

I am using a python 2.7 script that runs 24/7, I want a different log file produced by the logging module each time a loop is executed. Each file would have the timestamp has filename to avoid confusion.

So Far I got:

def main():

    while True:

        datetimenow = datetime.datetime.now().strftime("%Y-%m-%d-%H:%M:%S")

        logging.basicConfig(format='%(asctime)s %(levelname)-8s %(message)s',
                            datefmt='%a, %d %b %Y %H:%M:%S', filename="logs/" + datetimenow + '.log',
                            level=logging.INFO)
        logging.getLogger().addHandler(logging.StreamHandler())


        # ... start of action


    if __name__ == "__main__":

        main()

This produces one file and when the loop is started again it doesnt close and open a new file.

Also, seems that the console output is double printed as each line is outputted to console twice.

Any ideas to fix these ?

Martijn Pieters posted something about closing a logging file handler: https://stackoverflow.com/a/15474586/1513933 — Laurent LAPORTE, Sep 03 '17 at 20:06
what do you want exactly? do you want it to continue from the where it was or you want to have a new file in each loop? because you are giving the `finename` based on NOW datetime!! of course every second that passes you will have a new file. — Moher, Sep 03 '17 at 20:12
@Moher : I want a new file on each loop that continues/start the new logging sequence — user3202532, Sep 03 '17 at 20:31

score 1 · Answer 1 · answered Sep 04 '17 at 00:11

Ok, I got it working by removing the basicConfig snippet and building two handlers, one inside the loop for the file with different timestamp and one in the class for the console. The key is to remove the handler at the end of the loop, before adding it again with a different date. Here is the complete example:

import logging
import time
import datetime

logger = logging.getLogger('simple_example')
logger.setLevel(logging.INFO)

con = logging.StreamHandler()
con.setLevel(logging.INFO)

formatter = logging.Formatter('%(asctime)s %(levelname)-8s %(message)s')
con.setFormatter(formatter)
logger.addHandler(con)

def main():

a = 0

while True:

    datetimenow = datetime.datetime.now().strftime("%Y-%m-%d-%H:%M:%S")

    ch = logging.FileHandler("logs/" + datetimenow + '.log')
    ch.setLevel(logging.INFO)
    formatter = logging.Formatter('%(asctime)s %(levelname)-8s %(message)s')
    ch.setFormatter(formatter)
    logger.addHandler(ch)        

    time.sleep(5)

    a += 1

    logger.warning("logging step "+ str(b))

    time.sleep(5)

    logger.removeHandler(ch)

if __name__ == "__main__":

    main()

Sleep(5) is used for the purpose of testing and that it doesnt go too fast.

Python Logging different file on each turn

1 Answers1