9

I'm running gunicorn as:

guiconrn --bind=0.0.0.0:5000 --log-file gunicorn.log myapp:app

Seems like gunicorn.log keeps growing. Is there a way to specify a max size of the log file, so that if it reaches max size, it'll just override it.

Thanks!!

Paulo
  • 8,690
  • 5
  • 20
  • 34
abisko
  • 663
  • 8
  • 21
  • 2
    I don't see gunicorn setting related to max log size. You could use `logrotate`. https://docs.rackspace.com/support/how-to/understanding-logrotate-utility/ – zedfoxus Mar 17 '22 at 02:15
  • Question is from 2018, wow :-) I don't know any program on unix that do manage their own log file size. Buffers yes, but not log size. Always `logrotate`. – Daniel W. Mar 17 '22 at 10:54
  • I am not sure it is good practice, as log rotate seems to be such a commonly used tool. But I believe my python solution will work for you. – Paulo Mar 22 '22 at 20:17
  • https://stackoverflow.com/questions/36424335/how-to-perform-log-rotation-with-gunicorn – NicoNing Mar 23 '22 at 12:45

2 Answers2

0

TLDR;

I believe there might be a "python only" solution using the rotating file handler provided in the internal lib of python. (at least 3.10)

To test

I created a pet project for you to fiddle with:

  1. Create the following python file

test_logs.py

import logging
import logging.config
import time

logging.config.fileConfig(fname='log.conf', disable_existing_loggers=False)


while True:
    time.sleep(0.5)
    logging.debug('This is a debug message')
    logging.info('This is an info message')
    logging.warning('This is a warning message')
    logging.error('This is an error message')
    logging.critical('This is a critical message')
  1. Create the following config file

log.conf

[loggers]
keys=root

[handlers]
keys=rotatingHandler

[formatters]
keys=sampleFormatter

[logger_root]
level=DEBUG
handlers=rotatingHandler

[handler_rotatingHandler]
class=logging.handlers.RotatingFileHandler
level=DEBUG
formatter=sampleFormatter
args=('./logs/logs.log', 'a', 1200, 1, 'utf-8')

[formatter_sampleFormatter]
format=%(asctime)s - %(name)s - %(levelname)s - %(message)s
  1. Create the ./logs directory
  2. Run python test_logs.py

To Understand

As you may have noticed already, the setting that allow for this behaviour is logging.handlers.RotatingFileHandler and the provided arguments args=('./logs/logs.log', 'a', 1200, 10, 'utf-8')

RotatingFileHandler is a stream handler writing to a file. That allow for 2 parameters of interest:

  • maxBytes set arbitrarily at 1200
  • backupCount set arbitrarily to 10

The behaviour is that upon reaching 1200 Bytes in size, the file is closed, renamed to /logs/logs.log.<a number up to 10> and a new file is opened.

BUT is any of maxBytes or backupCount is 0. No rotation is done !

In Gunicorn

As per the documentation you can feed a config file.

This could look like:

guiconrn --bind=0.0.0.0:5000 --log-config log.conf myapp:app

You will need to tweak it to your existing setup.

Paulo
  • 8,690
  • 5
  • 20
  • 34
  • Acturally, I have been using `RotatingFileHandler` since Python>3.3 , I just put my draft code in my answer. – NicoNing Mar 23 '22 at 13:06
0

On Ubuntu/Linux, suggest to use logrotate to manage your logs, do like this: https://stackoverflow.com/a/55643449/6705684

Since Python>3.3, With RotatingFileHandler, here is my solution(MacOS/Windows/Linux/...) :

import os
import logging
from logging.handlers import RotatingFileHandler
fmt_str = '[%(asctime)s]%(module)s - %(funcName)s - %(message)s'
fmt = logging.Formatter(fmt_str)

def rotating_logger(name, fmt=fmt,
                      level=logging.INFO,
                      logfile='.log',
                      maxBytes=10 * 1024 * 1024,
                      backupCount=5,
                      **kwargs
                      ):
    logger = logging.getLogger(name)

    hdl = RotatingFileHandler(logfile, maxBytes=maxBytes, backupCount=backupCount)
    hdl.setLevel(level)
    hdl.setFormatter(fmt)

    logger.addHandler(hdl)
    return logger

more refer:

NicoNing
  • 3,076
  • 12
  • 23