Rotating mongrel production logs

In another thread, I had posted that we rotate our mongrel production
logs once a week as a means to keep the performance degradations that
happen with large log files at bay. A few of you asked me to post how
we do it, so here it is. We run our sites on RedHat Enterprise 4
machines. Our logrotate.conf file looks like this:

see “man logrotate” for details

rotate log files weekly

weekly

keep 4 weeks worth of backlogs

rotate 4

create new (empty) log files after rotating old ones

create

uncomment this if you want your log files compressed

#compress

RPM packages drop log rotation information into this directory

include /etc/logrotate.d

no packages own wtmp – we’ll rotate them here

/var/log/wtmp {
monthly
create 0664 root utmp
rotate 1
}

system-specific logs may be also be configured here.

mongrel logs

/path/to/rails/root/shared/log/*.log {
copytruncate
weekly
missingok
dateext
compress
sharedscripts
olddir /path/to/rails/root/old_mongrel_logs
rotate 28
postrotate
for i in ls /path/to/rails/root/shared/log/mongrel*.pid; do
kill -USR2 cat $i
done
endscript
}

Incidentally, we also use monit to monitor our various processes
(mongrel, Apache, ferretDRb MySQL, etc), so if for some odd reason
this log rotation would not bring one of our mongrels back up cleanly,
we’d be alerted and monit would try to restart it. So far, this has
not happened at all across any of our 6 production setups just like
this.

None of the sites are extremely high traffic, so it’s possible that
you’d want to rotate mongrel log files more frequently than weekly.

I hope this helps.

Sean B.
[email protected]

On Thu, Feb 14, 2008 at 11:29 AM, Sean B.
[email protected] wrote:

    postrotate
      for i in `ls /path/to/rails/root/shared/log/mongrel*.pid`; do
        kill -USR2 `cat $i`
      done
    endscript

Incidentally, we also use monit to monitor our various processes
(mongrel, Apache, ferretDRb MySQL, etc), so if for some odd reason
this log rotation would not bring one of our mongrels back up cleanly,
we’d be alerted and monit would try to restart it.

Why not ask monit to restart your mongrels? It seems kind of nice to
have monit be the only guy in town touching the processes it
monitors… but maybe there’s a technical reason why not?

-Nate

On Thu, Feb 14, 2008 at 2:08 PM, Nate V. [email protected] wrote:

this log rotation would not bring one of our mongrels back up cleanly,
we’d be alerted and monit would try to restart it.

Why not ask monit to restart your mongrels? It seems kind of nice to
have monit be the only guy in town touching the processes it
monitors… but maybe there’s a technical reason why not?

-Nate

No technical reason. Simply timing. Monit wakes itself every X
minutes to check if processes are running, so there could be a delay
of up X minutes until the mongrel process was restarted. The approach
in the logrotate restarts the mongrels immediately. Interestingly,
monit does realize that the process ID for each mongrel has changed
and alerts of that fact, but since we know that will happen every
Sunday at 3:00 AM, we can safely ignore it.

Sean B.
[email protected]

On Thu, Feb 14, 2008 at 1:44 PM, Sean B. [email protected]
wrote:

On Thu, Feb 14, 2008 at 2:08 PM, Nate V. [email protected] wrote:

Why not ask monit to restart your mongrels? It seems kind of nice to
have monit be the only guy in town touching the processes it
monitors… but maybe there’s a technical reason why not?

No technical reason. Simply timing. Monit wakes itself every X
minutes to check if processes are running, so there could be a delay
of up X minutes until the mongrel process was restarted. The approach
in the logrotate restarts the mongrels immediately. Interestingly,
monit does realize that the process ID for each mongrel has changed
and alerts of that fact, but since we know that will happen every
Sunday at 3:00 AM, we can safely ignore it.

I’m pretty sure that doing:

monit restart all -g your_mongrel_group

will wake monit up and have it restart your app before hitting its
timeout. This also avoids the worried emails from monit; ignoring ‘PID
changed’ messages from monit (even if you’re expecting them) is a bad
habit to get into.

-n

Umm, why are you restarting your mongrels at all?

copytruncate allows the processes to write to the log file without
restarting

The only issue is you may lose a few requests while it’s rotating, but
in my case, I’m using the nginx logs to provide the stats anyway.