Python File Handler with Rotating Content of File

Question

I have written a simple logging program that attaches anything I send to it to a file:

def log(message):
    with open ("log.txt", 'a+') as f:
        f.write(message + "\n")

However, I would like to limit how big this file gets. When it gets to the maximum size, I would like for it to remove the first lines and append at the bottom.

Is this possible with a file handler or do I need to code it myself? I am also fine using a rotating file handler, but all the examples I have seen let the environment write exceptions automatically after setting a level, and I need to control what is written to the file.

Many thanks in advance!

Is there a particular reason you need or want to use your own logging? You could use the logging modules rotating file handler if the answer is no: https://docs.python.org/3/library/logging.handlers.html#rotatingfilehandler — PirateNinjas, Aug 20 '20 at 14:12
@PirateNinjas - Thanks for replying. There is not a particular reason, but I would like to control what is written in that log as I need to write my own content, not only system exceptions. If this is possible with a rotating file handler, could you give me an example of how to modify my code to get this done and limit the number of files and file size? Thanks in advance! — Irina, Aug 20 '20 at 14:19
This answer https://stackoverflow.com/a/24505345/8237877 provides an example of what you're after. Basically, you define the handler, then log in the places you want to log. It shouldn't capture other system exceptions. — PirateNinjas, Aug 20 '20 at 14:26
@PirateNinjas - For the code provided in that question, if I understand correctly, I would include that code in a function (except the while loop at the end) and have the function return app_log, then use app_log.info(my_text) whenever I want to log something. Is that correct? — Irina, Aug 20 '20 at 15:01
yes, you could do that. Either that or just define it in the file directly, which would also work — PirateNinjas, Aug 20 '20 at 16:13
@PirateNinjas - Can you give me an example to define it in the file directly? Thanks again! — Irina, Aug 20 '20 at 17:13

score 5 · Accepted Answer · answered Aug 20 '20 at 17:43

This is an example of using python's built in RotatingFileHandler:

import logging
from logging.handlers import RotatingFileHandler

# change to a file you want to log to
logFile = 'log_r.log'

my_handler = RotatingFileHandler(logFile, mode='a', maxBytes=5*1024*1024,
                                 backupCount=2, encoding=None, delay=0)
my_handler.setLevel(logging.INFO)

app_log = logging.getLogger('root')
app_log.setLevel(logging.INFO)
app_log.addHandler(my_handler)


def bad():
    raise Exception("Something bad")


if __name__ == "__main__":
    app_log.info("something")
    try:
        app_log.info("trying to run bad")
        bad()
    except Exception as e:
        app_log.info("That was bad...")
    finally:
        app_log.info("Ran bad...")

The behaviour is slightly different to your proposed behaviour as it doesn't delete from the start of the file, instead moving the file to a different filename and starting from scratch.

Note that the only things that show in the log file when you run this are the pieces of text we're logging explicitly - i.e. no system junk you don't want.

Python File Handler with Rotating Content of File

1 Answers1