0

In the flask server which runs 24/7, I set up the TimedRotatingFileHandler to rotate the log file at everyday midnight. However the log rotation is delayed till something new is written to the log file after the midnight. How to fix this issue?

Here is the code I've used for log rotation.

config = ConfigParser.ConfigParser()
config.read("config.cnf")

#Logging
log_path = config.get('logging', 'log_path')

from logging.handlers import TimedRotatingFileHandler 

log = logging.getLogger(__name__)
log.setLevel(logging.DEBUG)
# add a file handler
fh = logging.handlers.TimedRotatingFileHandler(log_path,when='midnight',interval=1,backupCount=0)
fh.setLevel(logging.DEBUG)
# create a formatter and set the formatter for the handler.
frmt = logging.Formatter('%(asctime)s - %(levelname)s - %(message)s')
fh.setFormatter(frmt)
# add the Handler to the logger
log.addHandler(fh)
kane.zorfy
  • 1,000
  • 4
  • 14
  • 27
  • Why is this behaviour an issue? – DisappointedByUnaccountableMod Sep 16 '15 at 07:30
  • @barny I've set up a cron job to read the previous day's log file at some time after midnight to do some processing. Since the log write is delayed, there wont be any log file for yesterday to do processing. – kane.zorfy Sep 16 '15 at 07:55
  • Are you saying the log handler doesn't name the file appropriately until it closes it? What is the error you are getting? – DisappointedByUnaccountableMod Sep 16 '15 at 09:50
  • @barny The log handler does name the file correctly. Its just that the file rotation operation is delayed till something is written to the base log file. Since I also have a cron job which assumes that a log file for yesterday will be generated at midnight for sure, is there a way to rotate the log file even if nothing new comes up to the handler just after midnight. – kane.zorfy Sep 16 '15 at 10:07
  • Force something (an app in the flask server) to write to the log at 00:01? Your logfile reader can consume the logfile for yesterday once midnight has passed because no more log info will be written into it, can't you? – DisappointedByUnaccountableMod Sep 16 '15 at 10:31
  • Did you check http://stackoverflow.com/questions/3496727/why-doesnt-my-timedrotatingfilehandler-rotate-at-midnight - i.e. is your logging process actually running at midnight? – DisappointedByUnaccountableMod Sep 16 '15 at 10:48
  • I tried manually writing to the log file at just after midnight. But rotation is not happening. It seems like the write operation should be done from the same log handler. – kane.zorfy Sep 16 '15 at 10:49
  • I already some across the post. In that case, the process is not running at all time for this to happen – kane.zorfy Sep 16 '15 at 10:51
  • Manually writing won't affect it - as you said, something in your app has to log something to make the loghandler do the rotation. Still don't understand why you can't consume the logfile as-is, though. Or maybe something can call doRollover() on the logging handler? – DisappointedByUnaccountableMod Sep 16 '15 at 10:54
  • The reason I cant consume the log file directly is that what happens if nothing new is written to the log for one whole day?. The next day also I'll be using the same data – kane.zorfy Sep 16 '15 at 10:59
  • You know what day today is, so you can tell which is yesterday's file from the filename. or maybe you should change away from the default filenaming - see http://stackoverflow.com/questions/24649789/how-to-force-a-rotating-name-with-pythons-timedrotatingfilehandler and http://stackoverflow.com/questions/338450/timedrotatingfilehandler-changing-file-name – DisappointedByUnaccountableMod Sep 16 '15 at 11:00
  • I gave the explanation for consuming the base log file before rotation happens. I think maybe we can extend the logging class to get the desired behavior. – kane.zorfy Sep 16 '15 at 11:03

0 Answers0