Rails app going nuts at 100% cpu

Damn it failed again! So I have to restart the hunt from the scratch, I
might have not waited long enough to let it fail…

Hi, my app got fucked again!

So it might not be acts_as_tree’s fault. Reading my log book and my
repository commits, I am now having high doubts about rcov_plugin: even
though it’s not loaded in production.rb, it’s still sitting in
vendor/plugins and it might be causing a problem.

I’ll report tomorrow morning.

Fernando P. wrote:

Damn it failed again! So I have to restart the hunt from the scratch, I
might have not waited long enough to let it fail…

Some things to help with a frozen app:

strace
http://eigenclass.org/hiki/ruby+live+process+introspection
http://github.com/rogerdpack/live-console/tree/master
http://www.ruby-forum.com/topic/191210
GL!
-r

My app is still failing. I installed the MemoryLogic plugin to check the
memory usage after each request, but it’s stable.

I noted that it’s a single request that locks the app as it doesn’t get
printed in the production.log file. Is it possible to debug a live
running Rails app? I looked at LiveConsole but it’s only for ruby apps
not rails.

A copy of XX has been removed from the module tree but is still active!

Usually [for me] that means “you’re running in development node not
production”

Could that be related to some memory leak that will eventually lock a
rails process at 100% cpu after some time?

might be related to
http://redmine.ruby-lang.org/issues/show/1484

Hi Fernando,

On Fri, 2009-08-21 at 09:49 +0200, Fernando P. wrote:

My app is still failing. I installed the MemoryLogic plugin to check the
memory usage after each request, but it’s stable.

I noted that it’s a single request that locks the app as it doesn’t get
printed in the production.log file. Is it possible to debug a live
running Rails app? I looked at LiveConsole but it’s only for ruby apps
not rails.

Sorry to hear you haven’t gotten this sorted out yet.

By way of alternatives, here’s a potentially productive debugging
approach.

Put production logging into debug mode by uncommenting the following
line in environment.rb:

config.log_level = :debug

Your logs will be much bigger so you’ll want to clear them on a daily
basis. But do not just delete them. Save them somewhere first.

You say “it doesn’t get printed in the production.log file”. That’s ok.
My guess is that the problem you’re having is almost surely
state-related. The next time your app fails, walk the log backwards to
figure out what state your app was in just prior to the failure. It’s
not what we’re shooting for, but in general most Rails apps aren’t
stateless. You can narrow the list of requests that might be the
problem by figuring out what state the app was in prior to the
failure.

You’re in production mode so I assume you’ve got multiple users on the
system. That means you’ll have to sort them out. Write a little script
to do that by IP. Then start doing your analysis. Do it on both sets
of logs: ones where the app failed, but especially the ones where the
app did not fail. Your analysis needs to ask: “what’s the
difference?” What was each user doing just before the problem. Were
all the users in one part of the app? Or did one just do something new
while all the rest were doing X and Y ?

Debugging a running production app is not impossible, but it’s damn
close to impossible IME. The best I can suggest is start gathering as
much info as possible and doing some heavy analysis to narrow down the
possibilities.

Best of luck,
Bill

Hi,

I finally solved my problem. It was a regular expression that failed on
a specific URL which locked the process! I never thought such thing
could happen!

Thank you all for your support and debugging ideas.

Best regards,

Fernando P. wrote:

Hi,

I finally solved my problem. It was a regular expression that failed on
a specific URL which locked the process! I never thought such thing
could happen!

What was the exact regex and ruby version and example?
Thanks!
-r

check your yml config files

maybe you mixed ident spaces with a tab ?

I’ve been reading all along but had nothing to offer to help. I’m very
glad you got it solved. :slight_smile:

On Aug 21, 11:22 am, Fernando P. [email protected]