Enabling Fluentd Log rotation

Question

I am using fluentd with the tg-agent installation. And I observed my default td-agent.log file is growing without having any log rotation.

I am using the following command to run the td-agent

/etc/init.d/td-agent start

And I found the following link which tells how to configure the rotation and it seems like this is with the fluent itself.

https://docs.fluentd.org/deployment/logging

anyone knows how to configure the rotation with the command I am using? I have the td-agent config file also.

You ought to configure and try out the configuration according to your requirements. It should work for `td-agent` as well. — Azeem, Aug 08 '20 at 09:54

score 2 · Answer 1 · answered Aug 10 '20 at 07:02

You can do this in two ways , first with td-agent itself and for this you need to update the td-agent init file /etc/init.d/td-agent. you have to find the below line in the file

TD_AGENT_ARGS="${TD_AGENT_ARGS:-${TD_AGENT_BIN_FILE} --log ${TD_AGENT_LOG_FILE} ${TD_AGENT_OPTIONS}}"

and update it to

TD_AGENT_ARGS="${TD_AGENT_ARGS:-${TD_AGENT_BIN_FILE} --log-rotate-age 5 --log-rotate-size 104857600 --log ${TD_AGENT_LOG_FILE} ${TD_AGENT_OPTIONS}}"

then restart td-agent and the result will be as shown below

16467 /opt/td-agent/embedded/bin/ruby /usr/sbin/td-agent --log-rotate-age 5 --log-rotate-size 104857600 --log /var/log/td-agent/td-agent.log --use-v1-config --group td-agent --daemon /var/run/td-agent/td-agent.pid
16472 /opt/td-agent/embedded/bin/ruby -Eascii-8bit:ascii-8bit /usr/sbin/td-agent --log-rotate-age 5 --log-rotate-size 104857600 --log /var/log/td-agent/td-agent.log --use-v1-config --group td-agent --daemon /var/run/td-agent/td-agent.pid --

The second method is to use logrotate for rotating the logs, create the below file on your server and make sure that logrotate is installed and it will take care of rotating the logs

 cat /etc/logrotate.d/td-agent
/var/log/td-agent/td-agent.log {
  daily
  rotate 30
  compress
  delaycompress
  notifempty
  create 640 td-agent td-agent
  sharedscripts
  postrotate
    pid=/var/run/td-agent/td-agent.pid
    if [ -s "$pid" ]
    then
      kill -USR1 "$(cat $pid)"
    fi
  endscript
}

Enabling Fluentd Log rotation

1 Answers1