Create new log file for everyday using python logging

Question

My python code works fine to create a log file with name and date on which code was executed -

For example-

I run code today it will create log file - logfile_2020-01-15.log

I run code tomorrow it will create log file - logfile_2020-01-16.log and so on.

Now the concern is, if my code execution is started today and it keeps on running for 8 days. It should create 8 log files - 1 file each day: logfile_2020-01-15.log to logfile_2020-01-23.log

But this is not happening. It keeps on logging in same file: logfile_2020-01-15.log when the code was initiated.

Please can anyone help me modify code-

import datetime
import logging
import schedule
class Workflow:    

    def setupLoggingToFile():
        logFilePath = "C:\ExceptionLogFiles\"
        logdate = datetime.datetime.now().strftime('%Y-%m-%d')
        logging.basicConfig(
            format='%(asctime)s %(name)-12s %(levelname)-8s %(message)s',
            datefmt='%m-%d-%y %H:%M:%S',
            level=logging.DEBUG,
            handlers=[RotatingFileHandler(logFilePath + "logfile_"+logdate+".log",maxBytes=10485760, backupCount=100)])   

    def StartWorkflow(self):
        try:
            print("New Cron Cycle Started..")
        except Exception:
            logging.exception("Something went wrong.", exc_info=True)

    def StartCron(self):
        try:
            schedule.every(5).seconds.do(self.StartWorkflow)
            while 1:
                schedule.run_pending()
                time.sleep(1)
        except Exception:
            logging.debug("CRON was unable to start. Something Wrong in StartCron function.")
            logging.exception("CRON was unable to start. Something Wrong in StartCron function.", exc_info=True)


A = Workflow()
A.StartCron()

Couple things that's missing from your code. 1) You forgot in include "self" in setupLoggingToFile. 2) The RotatingFileHandler only rotate logs once a certain size is reached, in your case maxBytes = 10485760. What you should be using is TimedRotatingFileHandler and set it to midnight. See https://docs.python.org/3/library/logging.handlers.html. 3) Why is handlers a list? — dreamzboy, Jan 16 '20 at 02:12

score 2 · Accepted Answer · answered Jan 16 '20 at 02:45

I would clean up the code a little bit and re-write it this way. New logs are rotated and appended the date/time at the end of the file.

>>> def logSetup ():
...    logger = logging.getLogger('testlog')
...    logger.setLevel(logging.DEBUG)
...    formatter = logging.Formatter(fmt='%(asctime)s %(name)-12s %(levelname)-8s %(message)s',
...                                  datefmt='%m-%d-%y %H:%M:%S')
...    fh = TimedRotatingFileHandler('/Documents/projects/python/testlog.log', when='S', interval=5)
...    fh.setFormatter(formatter)
...    logger.addHandler(fh)
...    return logger

In your case, you would need to change the following line to:

fh = TimedRotatingFileHandler('/Documents/projects/python/testlog.log', when='midnight')

Writing to log file and simulating log rotate every 5 seconds.

>>> for i in range (20):
...    logger.debug('%d Writing some logs' % i)
...    sleep (1)
...

The results become:

$ cat testlog.log.2020-01-15_18-35-01
01-15-20 18:36:24 testlog      DEBUG    0 Writing some logs
01-15-20 18:36:25 testlog      DEBUG    1 Writing some logs
01-15-20 18:36:26 testlog      DEBUG    2 Writing some logs
01-15-20 18:36:27 testlog      DEBUG    3 Writing some logs
01-15-20 18:36:28 testlog      DEBUG    4 Writing some logs
$ cat testlog.log.2020-01-15_18-36-24
01-15-20 18:36:29 testlog      DEBUG    5 Writing some logs
01-15-20 18:36:30 testlog      DEBUG    6 Writing some logs
01-15-20 18:36:31 testlog      DEBUG    7 Writing some logs
01-15-20 18:36:32 testlog      DEBUG    8 Writing some logs
01-15-20 18:36:33 testlog      DEBUG    9 Writing some logs
$ cat testlog.log.2020-01-15_18-36-29
01-15-20 18:36:34 testlog      DEBUG    10 Writing some logs
01-15-20 18:36:35 testlog      DEBUG    11 Writing some logs
01-15-20 18:36:36 testlog      DEBUG    12 Writing some logs
01-15-20 18:36:37 testlog      DEBUG    13 Writing some logs
01-15-20 18:36:38 testlog      DEBUG    14 Writing some logs
$ cat testlog.log.2020-01-15_18-36-34
01-15-20 18:36:39 testlog      DEBUG    15 Writing some logs
01-15-20 18:36:40 testlog      DEBUG    16 Writing some logs
01-15-20 18:36:41 testlog      DEBUG    17 Writing some logs
01-15-20 18:36:42 testlog      DEBUG    18 Writing some logs
01-15-20 18:36:43 testlog      DEBUG    19 Writing some logs
$ cat testlog.log
01-15-20 18:36:39 testlog      DEBUG    15 Writing some logs
01-15-20 18:36:40 testlog      DEBUG    16 Writing some logs
01-15-20 18:36:41 testlog      DEBUG    17 Writing some logs
01-15-20 18:36:42 testlog      DEBUG    18 Writing some logs
01-15-20 18:36:43 testlog      DEBUG    19 Writing some logs

Also, just curious, the log files are created with naming: `testlog.log.2020-01-15_18-35-01`.Is it posible to create naming like: `testlog_2020-01-15_18-35-01.log` ? — user2961127, Jan 16 '20 at 15:41
Having the time appended to the end of the file is quite a common practice and an industry standard but I guess for Windows it's different. Unfortunately, TimedRotatingFileHandler doesn't give you such option. It's hardcoded in the library. A dirty hack would be give your log file a name like testlog (without .log), let the filehandler append the suffix, replace the period in the file name with underscore, and finally go back and add the ".log" at the end of the file. — dreamzboy, Jan 16 '20 at 20:36

Create new log file for everyday using python logging

1 Answers1

Linked