8

Logback version 1.2.3

I want to use a SizeAndTimeBasedRollingPolicy in our logback configuration file (logback.xml), but at this time the SizeAndTimeBasedRollingPolicy doesn't function as expected. (https://logback.qos.ch/manual/appenders.html#SizeAndTimeBasedRollingPolicy)

Ideally I want to keep logs dating back no later than ex. 90 days, each file no larger than 100MB and a total archive size of ex. 10GB in total.

As it currently stands, the totalSizeCap is applied to each entry within the MaxHistory range. Ex.

<appender name="ROLLING" class="ch.qos.logback.core.rolling.RollingFileAppender">
    <file>mylog.txt</file>
    <rollingPolicy class="ch.qos.logback.core.rolling.SizeAndTimeBasedRollingPolicy">
      <!-- rollover daily -->
      <fileNamePattern>mylog-%d{yyyy-MM-dd}.%i.txt</fileNamePattern>
       <!-- each file should be at most 100MB, keep 60 days worth of history, but at most 20GB -->
       <maxFileSize>100MB</maxFileSize>    
       <maxHistory>60</maxHistory>
       <totalSizeCap>1GB</totalSizeCap>
    </rollingPolicy>
    <encoder>
      <pattern>%msg%n</pattern>
    </encoder>
</appender>

The above XML configuration will create logs spanning over 60 days, applying a totalSizeCap of 1GB per day. This will lead to a total archive size of 60GB instead of the expected 1GB. If the totalSizeCap is reached during the day the day's logs will start to rollover by deleting the day's oldest files, this will create gaps within the log history, which we don't want. A work-around for this error was to use a yearly rollover, instead of daily or monthly rollover, unfortunately yearly rollover doesn't work when using a SizeAndTimeBasedRollingPolicy.

Does anyone know of this problem, has this been fixed, or am I doing something wrong in my configuration?

Iggydv
  • 166
  • 2
  • 12

1 Answers1

2

It seems that you have found a bug!

There should be no gaps in the logs. When totalSizeCap is reached, the oldest log file should be deleted. When maxHistory reached, the oldest log file should be deleted.

Unfortunately it seems that there is a bug in logback which causes gaps in the logging because not the oldest files are deleted. See demonstration here: https://github.com/riskop/slf4j_logback_SizeAndTimeBasedRollingPolicy_problem

I've opened an issue: https://jira.qos.ch/browse/LOGBACK-1361

According to Gülcü it will be fixed in logback classic 1.3.0.

Note that you can "vote" the issue on logback's Jira!

riskop
  • 1,693
  • 1
  • 16
  • 34
  • 2
    This doesn't seem to be the case for me though. If I for example set MaxHistory to 1 (1 day) and totalsizeCap of 1GB then it will save approximately 1 GB's worth of 100MB files for the previous day. During the next day, the previous day's files aren't deleted and the appender only rolls over on files with the current date. It looks something like this `core-2017-12-12.01.log` `...` `core-2017-12-12.10.log` `core-2017-12-13.03.log` `...` `core-2017-12-13.13.log` as you can see the appender rolls over on todays log files but leaves anything within the maxHistory untouched – Iggydv Dec 13 '17 at 12:20
  • 2
    I think I can reproduce your problem, at least when the rolling interval is very short (roll is in every second ). My test is here: https://github.com/riskop/slf4j_logback_SizeAndTimeBasedRollingPolicy_problem Could you check it in your environment? I've tried it on xubuntu 16, openjdk8. – riskop Dec 15 '17 at 10:10
  • 2
    I've created an issue for logback: https://jira.qos.ch/browse/LOGBACK-1361 – riskop Dec 15 '17 at 10:26
  • Thank you very much for creating the issue! – Iggydv Dec 18 '17 at 12:17
  • Any chance to have this fixed in 1.2.x branch? – Sergei Rodionov Mar 14 '19 at 11:46
  • @SergeiRodionov : I think you should ask Ceki – riskop Mar 14 '19 at 12:45
  • That would be last thing on my mind. I don't want to put more burden on OSS maintainers. Just venting... – Sergei Rodionov Mar 14 '19 at 13:32