how to create multiple log files in a log folder when it exceeds the maxsize

Question

I have a program that give a logging information in log file but now I have created a folder named as LogFolder I'm keeping my log file at the same folder but I want to create every time a new file when its increases the maxBytes size with different name

My log file is written in json format if you know normal format also for same query then you can help me with that.

My logging.json file is:

{
  "version": 1,
  "disable_existing_loggers": false,
  "formatters": {
        "json": {
        "format": "%(asctime)s - %(name)s - %(levelname)s - %(message)s",
            "()": "pythonjsonlogger.jsonlogger.JsonFormatter"
        }
  },
  "handlers": {
        "console": {
            "class": "logging.StreamHandler",
            "level": "DEBUG",
            "formatter": "json",
            "stream": "ext://sys.stdout"
        },
        "file_handler": {
            "class": "logging.handlers.RotatingFileHandler",
            "level": "DEBUG",
            "formatter": "json",
            "filename": "..\\LogFloder\\Data.log",
            "mode": "a",
            "maxBytes": 25600,
            "encoding": "utf8"
         }
  },
  "root": {
       "level": "DEBUG",
       "handlers": ["console", "file_handler"]
  }
}

This is how I call it in my python file main.py:

import logging.config
import json
fp = open('logging.json')
logging.config.dictConfig(json.load(fp))
logging.getLogger("requests").setLevel(logging.WARNING)
logger = logging.getLogger(__name__)
logger.removeHandler(default_handler)
fp.close()

Here all the things are working very fine. I just want to make new log file with different name in LogFolder when its exceeds the maxbytes size ...please help me how to do it.

Thanks in advance..

why do you need to change the name of the new log file ? see documentation for RotatingFileHandler, it is already handling the file names for you once the maxBytes is reached **if** you specify a value for the backupCount parameter. — , Dec 07 '18 at 11:22
This post might help: https://stackoverflow.com/questions/24505145/how-to-limit-log-file-size-in-python — ak_app, Dec 07 '18 at 11:25
its working its creating new file i have written like "maxBytes": 6144, "backupCount": 30, but now after 2 files its directly going to file name like data10.log ...... its should show data3.log @ak_app — snehil singh, Dec 07 '18 at 12:16
@reportgunner my bad its creating new file thnx but have another problem which i have written in 2nd msg please see that.. — snehil singh, Dec 07 '18 at 12:18
@snehil singh make sure that something is not locking the files (maybe you are reading the file via notepad ?) I've just tested with maxBytes=1 and backupCount=30 and everything works fine — , Dec 07 '18 at 13:08

buran · Answer 1 · 2018-12-12T07:52:42.453

Have a look at RotatingFileHandler

EDIT to expand the answer as per recommendation from comment

The RotatingFileHandler class supports rotation of disk log files. At instantiation you can supply two optional arguments - maxBytes with default value of 0 and backupCount with default value of 0

You can use the maxBytes and backupCount values to allow the file to rollover at a predetermined size. When the size is about to be exceeded, the file is closed and a new file is silently opened for output. Rollover occurs whenever the current log file is nearly maxBytes in length; but if either of maxBytes or backupCount is zero, rollover never occurs, so you generally want to set backupCount to at least 1, and have a non-zero maxBytes. When backupCount is non-zero, the system will save old log files by appending the extensions ‘.1’, ‘.2’ etc., to the filename. For example, with a backupCount of 5 and a base file name of app.log, you would get app.log, app.log.1, app.log.2, up to app.log.5. The file being written to is always app.log. When this file is filled, it is closed and renamed to app.log.1, and if files app.log.1, app.log.2, etc. exist, then they are renamed to app.log.2, app.log.3 etc. respectively.

There is also TimeRotatingFileHandler that allow to rotate log files based on time.

how to create multiple log files in a log folder when it exceeds the maxsize

1 Answers1