MongoDB log file growth

Question

Currently my log file sits at 32 meg. Did I miss an option that would split the log file as it grows?

Mine grew to 23GB after being left unattended for a year and blew up the server :O — Elad Nava, Jan 05 '20 at 21:23

score 58 · Answer 1 · edited May 15 '22 at 05:27

58

You can use logrotate to do this job for you.

Put this in /etc/logrotate.d/mongod (assuming you use Linux and have logrotate installed):

/var/log/mongo/*.log {
    daily
    rotate 30
    compress
    dateext
    missingok
    notifempty
    sharedscripts
    copytruncate
    postrotate
        /bin/kill -SIGUSR1 `cat /var/lib/mongo/mongod.lock 2> /dev/null` 2> /dev/null || true
    endscript
}

edited May 15 '22 at 05:27

oguz ismail

1
16
47
69

answered Dec 06 '11 at 06:45

pdwalker

823
7
8

Just FYI, I installed from 10gen's official rpms and lograte was configured automatically. – Nic Cottrell Dec 06 '11 at 15:17
Useful to know. The installations I've previously done didn't have that. – pdwalker Mar 28 '12 at 09:28
7

Thanks for the config file. Be sure to check the paths in this script, though - in my case that was /var/log/mongodb/ and /var/lib/mongodb . – Erwin Wessels Jul 02 '13 at 07:25
3

What is happening in the postrotate. Is this something I should be wary of implementing on a production system? – Matt Canty Jul 02 '13 at 15:22
3

@Matthew: check the docs (linked in the accepted answer). Short version: it causes mongodb to close the previous log and open the next one. Due to Unix file semantics, if you don't do that, mongo will keep writing to the previous file. – cha0site Aug 15 '13 at 12:33
The above logrotate script needs `copytruncate` to avoid crashing MongoDB versions 2.4.7 and 2.4.6. See https://jira.mongodb.org/browse/SERVER-11087 for details. – hvrauhal Oct 31 '13 at 09:19
Thanks. This sample script along with `logrotate` man page was very useful and educating for me. Also, as pointed out by @ErwinWessels *people must pay attention to the paths.* – Luciano Jan 22 '14 at 12:00
May i know the meaning of `/bin/kill -SIGUSR1 cat /var/lib/mongo/mongod.lock 2> /dev/null 2> /dev/null || true` – fmchan May 07 '15 at 01:34
1

It sends a signal to the mongod process to tell it to close and reopen the log file. See here for more on signals: http://stackoverflow.com/questions/5350865/why-do-many-unix-programs-use-signals-like-usr1 – pdwalker May 07 '15 at 04:57
I dun know why there are two logs generated, mongo.log-20150511 and mongo.log.2015-05-11T20-02-03 and the second log has no data. since my data is in /export/home/d17/test/data/v1/mongo/repl0 so my postrotate script is /bin/kill -SIGUSR1 `cat /export/home/d17/test/data/v1/mongo/repl0/mongod.lock 2> /dev/null` 2> /dev/null || true – fmchan May 12 '15 at 06:17
I only need mongo.log-20150511, but why is mongo.log.2015-05-11T20-02-03 also generated? – fmchan May 12 '15 at 06:18
@fmchan - it's because there's 2 log rotation managers. First, logrotate runs, copying `mongod.log` to `mongod.log-20150511`. After that, the postrotate script is executed, and when mongod receives SIGUSR1 it also does a rotation, copying `mongod.log` (now empty) is then copied by mongod (not logrotate) to `mongod.log.2015-05-11T20-02-03`. You could add a line to your postrotate script to delete all mongod style rotated logs, after the kill command: `rm -f /path/to/log/mongo.log*T*` – Brett Jan 06 '16 at 13:08
1

p.s. you can control logrotate's naming pattern by using the `dateformat` parameter - see the logrotate manual for more info. http://linux.die.net/man/8/logrotate – Brett Jan 06 '16 at 13:10
Great answer. Couple notes. Don't send a SIGUSR1 to Monogo after moving its active file, it will coredump. Not that your config does that, but I did it while testing. Also this worked for me /bin/cat /var/run/mongodb/mongod.pid |xargs kill -SIGUSR1 2> /dev/null|| true Just a little more straight forward in my mind. – AaronM Jun 12 '17 at 16:08
1

@Brett There is a systemLog.logRotate property that you can use which supports logrotate and doesn't create the `mongod.log.2015-05-11T20-02-03`. Set logRotate to reopen and logAppend to true. See https://docs.mongodb.com/v3.4/tutorial/rotate-log-files/ – Andrew H Dec 07 '17 at 18:52

score 6 · Answer 2 · answered Feb 15 '11 at 17:31

6

If you think that 32 megs is too large for a log file, you may also want to look inside to what it contains.

If the logs seem mostly harmless ("open connection", "close connection"), then you may want to start mongod with the --quiet switch. This will reduce some of the more verbose logging.

answered Feb 15 '11 at 17:31

Gates VP

44,957
11
105
108

4

Or, as of 2.0, add `quiet = true` to your config file (`/etc/mongod.conf` on some systems). – Adam Monsen Jun 20 '12 at 19:05

score 1 · Answer 3 · edited Sep 26 '17 at 08:17

Using logrotate is a good option. while, it will generate 2 log files that fmchan commented, and you will have to follow Brett's suggestion to "add a line to your postrotate script to delete all mongod style rotated logs".

Also copytruncate is not the best option. There is always a window between copy and truncate. Some mongod logs may get lost. Could check logrotate man page or refer to this copytruncate discussion.

Just provide one more option. You could write a script that sends the rotate signal to mongod and remove the old log files. mongologrotate.sh is a simple reference script that I have written. You could write a simple cron job or script to call it periodically like every 30 minutes.

score 1 · Accepted Answer · answered Feb 15 '11 at 14:22

1

Rotate the logs yourself

http://www.mongodb.org/display/DOCS/Logging

or use 'logrotate' with an appropriate configuration.

answered Feb 15 '11 at 14:22

25

"with an appropriate configuration" : that is exactly the question... What is an appropriate configuration. Your answer (and MongoDB documentation) are not clear about this subject. – jmcollin92 Sep 16 '16 at 16:38

MongoDB log file growth

4 Answers4

Linked