Hey folks.
Using mongrel_rails, Is it possible to have multiple mongrel instances
write to the same log file? I suppose that implementing this might
incur a performance hit since each mongrel would need to lock and
unlock the file in order to not weave the log messages together in an
unreadable stream? Or perhaps all the log messages are single-line
anyway.
Thanks,
john
–
John Joseph B.
http://blog.johnjosephbachir.org
http://lyceum.ibiblio.org
On 2/5/08, John Joseph B. [email protected] wrote:
Hey folks.
Using mongrel_rails, Is it possible to have multiple mongrel instances
write to the same log file? I suppose that implementing this might
incur a performance hit since each mongrel would need to lock and
unlock the file in order to not weave the log messages together in an
unreadable stream? Or perhaps all the log messages are single-line
anyway.
Not 100 % sure I see what you’re looking for, but anyway: In my
out-of-the-box setup, my three mongrelled rails instances all write to
log/production.log. There is no locking, and sometimes log messages are
interleaved (interleft?).
The mongrels write a little something to their respective log files
during
start and stop, but that’s all.
You might want to have a look at Kirk H.'s “Analogger”, which I
believe
solves the interleaving problem. I remember Kirk advising us on it on
the
list, so you could have a look in the archive.
/David
On Feb 5, 2008 3:06 PM, John Joseph B. [email protected]
wrote:
Hey folks.
Using mongrel_rails, Is it possible to have multiple mongrel instances
write to the same log file? I suppose that implementing this might
incur a performance hit since each mongrel would need to lock and
unlock the file in order to not weave the log messages together in an
unreadable stream? Or perhaps all the log messages are single-line
anyway.
Are you talking about the mongrel log itself (which usually doesn’t
accumulate much), or your application’s log?
With as little as Mongrel itself writes, I wouldn’t see any
significant problem in just having all of your mongrels write to the
same file, personally.
If you are talking about application logs, then one way is to use an
external logging service of some sort. I have one, Analogger
(http://analogger.swiftcore.org). As of the release there it isn’t
quite drop in compatible with Rails, but I’m releasing an update which
provides an interface to allow drop in Rails compatibility.
Kirk H.
On Feb 5, 2008 5:19 PM, David V. [email protected] wrote:
Not 100 % sure I see what you’re looking for, but anyway: In my
out-of-the-box setup, my three mongrelled rails instances all write to
log/production.log.
On Feb 5, 2008 5:24 PM, Kirk H. [email protected] wrote:
Are you talking about the mongrel log itself (which usually doesn’t
accumulate much), or your application’s log?With as little as Mongrel itself writes, I wouldn’t see any
significant problem in just having all of your mongrels write to the
same file, personally.
Thanks for the quick replies. With a mongrel_cluster config file like
this:
user: root
log_file: /path/to/app/log/dir/mongrel.log
port: “8000”
environment: production
group: apache
address: 127.0.0.1
servers: 8
…
I end up with these log files:
mongrel.8000.log
mongrel.8001.log
mongrel.8002.log
mongrel.8003.log
mongrel.8004.log
mongrel.8005.log
mongrel.8006.log
mongrel.8007.log
With, as has been stated, very little written to each, and the
application logging going to production.log.
It would be nice if I could keep things tidier and have just one
mongrel.log
Thanks!
John
–
John Joseph B.
http://blog.johnjosephbachir.org
http://lyceum.ibiblio.org
We did this on “accident” on our site, and the performance hit is
horrible. You might not notice it when you’re not getting a lot of
traffic, but when you do, the performance hit is really awful. Each
mongrel needs to have its own log file and log only the messages you
absolutely need (which is pretty much true for every web server).
Cheers,
Kevin L.
On Feb 5, 2008 5:23 PM, Kevin L. [email protected] wrote:
We did this on “accident” on our site, and the performance hit is
horrible. You might not notice it when you’re not getting a lot of
traffic, but when you do, the performance hit is really awful. Each
mongrel needs to have its own log file and log only the messages you
absolutely need (which is pretty much true for every web server).
That doesn’t really make sense, though, since Mongrel logs almost
nothing. It shouldn’t materially affect performance because it is
almost never writing anything to the file.
If, on the other hand, you accidentally pooled your app logs into the
same file, then yeah, that will create a performance hit under load
because every requests on each different process is trying to push to
that same file and just depending on the OS to sort it out.
Kirk H.
On Feb 5, 2008 7:32 PM, Kirk H. [email protected] wrote:
If, on the other hand, you accidentally pooled your app logs into the
same file, then yeah, that will create a performance hit under load
because every requests on each different process is trying to push to
that same file and just depending on the OS to sort it out.
But the application log is already consolidated (to production.log) by
default-- are you suggesting that this incurs a performance hit and
the log should be split up?
–
John Joseph B.
http://blog.johnjosephbachir.org
http://lyceum.ibiblio.org
On Feb 6, 2008, at 16:53 , John Joseph B. wrote:
But the application log is already consolidated (to production.log) by
default-- are you suggesting that this incurs a performance hit and
the log should be split up?
At least, that’s the experiences we had with our Rails application;
random slowdowns for no apparent reason. All went away after we moved
to SyslogLogger.
–
Jakob S. - http://mentalized.net
I recently just blogged about my experiences with large logfiles
effecting mongrel. It’s not hugely scientific, but there is
definitely a connection between mongrel memory usage and the rails
production.log.
http://blog.edhickey.com/2008/02/08/mongrel-memory-usage/
On Feb 5, 2008, at 2:06 PM, John Joseph B. wrote:
john
Hi John,
You’ll want to use something like SyslogLogger.
Regards, Jason
On Thu, Feb 14, 2008 at 1:38 AM, Zed A. Shaw [email protected]
wrote:
how mongrel would cause that. It doesn’t touch the logs or logger or
anything and leaves that all to Rails.How much you want to bet that hack job rails-core did on the fucking
Logger class causes this? Not sure how. That’s just weird.Next would to see if it happens in jruby since they use a different IO
setup.Anyway, looks like syslogger is the way to go for production.
This won’t address the cause, but can alleviate the symptoms. We use
logrotate to rotate our production logs once a week and that
definitely helps keep things snappy. You certainly set them to rotate
more or less frequently if your application needs it. If anyone would
like to see our logrotate configuration, let me know.
–
Sean B.
[email protected]
On Mon, 11 Feb 2008 09:36:06 -0600
“Ed Hickey” [email protected] wrote:
I recently just blogged about my experiences with large logfiles
effecting mongrel. It’s not hugely scientific, but there is
definitely a connection between mongrel memory usage and the rails
production.log.
http://blog.edhickey.com/2008/02/08/mongrel-memory-usage/
That’s fucking weird. There’s definitely a correlation, but not sure
how mongrel would cause that. It doesn’t touch the logs or logger or
anything and leaves that all to Rails.
How much you want to bet that hack job rails-core did on the fucking
Logger class causes this? Not sure how. That’s just weird.
Next would to see if it happens in jruby since they use a different IO
setup.
Anyway, looks like syslogger is the way to go for production.
–
Zed A. Shaw
On Feb 13, 2008 10:50 AM, Sean B. [email protected] wrote:
If anyone would
like to see our logrotate configuration, let me know.
I would be interested in seeing your config, thanks for offering.
Perhaps it’s worthy of a new thread?
–
John Joseph B.
http://blog.johnjosephbachir.org
http://lyceum.ibiblio.org
At least in my case, the issue is definitely related to the size of
the log. All the sapped memory gets freed as soon as I truncate the
production.log.
I tried SyslogLogger today
(http://rails-analyzer.rubyforge.org/hacks/classes/SyslogLogger.html)
but couldn’t get the rails log to only show up in one place when
following the README’s directions It was logging to
/var/log/production.log as well as /var/log/messages. Server load
jumped up to 2, I assume because of all the disk IO.
Anyone who’s deployed SyslogLogger and has insight, comments welcome.
Also, anyone tried Analogger? http://analogger.swiftcore.org/
ed
So, it’s the size of the LOG that is causing the issue?
I’m rotating every night anyway using logrotate, but I was thinking
about moving to syslogger if I could find a nice how-to.
On 13 Feb 2008, at 21:56, Ed Hickey wrote:
Anyone who’s deployed SyslogLogger and has insight, comments welcome.
Also, anyone tried Analogger? http://analogger.swiftcore.org/
I’m not running it in production yet, but we’re using it in a
development environment for our application. I sent Kirk H. a
patch which implements a Logger style api which you can hand to the
rails log config options. I believe this will be coming with the
release that is coming up soon.
It’s been working just fine under development for our group, and we
will be going production with it shortly.
This forum is not affiliated to the Ruby language, Ruby on Rails framework, nor any Ruby applications discussed here.
Sponsor our Newsletter | Privacy Policy | Terms of Service | Remote Ruby Jobs