Hey guys,
I’m using JRuby in local development mode (because I intend to deploy
with it - yay threads!) and having one very odd issue that I wanted to
see if anyone else has run into before.
Basically I’m working on a Rails 3.2.11 app with the latest JRuby out
(1.7.2 as of now). If I issue something like “bundle exec rake spec” or
“rake db:migrate”, sometimes the operation finishes (usually, in fact),
and sometimes it doesn’t, but in about 90% of cases it gets nearly done,
and then right before actually exiting it just plain hangs there doing
nothing.
For example, I can throw down “rake db:migrate”, see the migration
happen, see the statements in development.log, access my PostgreSQL
console, \d and see that it certainly did work, but the
actual process is just hung indefinitely. I’ve been CTRL+C’ing out, but
this is definitely a bit odd.
I do also have drip installed to try to speed up load times, and I have
a suspicion that’s involved somehow, but before removing it (since it
really does help), I wanted to see if anyone else has run into this
issue or has any ideas on what I can do to keep drip and just get rid of
this odd behavior.
Thanks for any help you can offer!
Hi,
When you get the hang I would use jstack to dump threads for the java
process. This should give you an idea of what’s going on when it’s hung.
I’ve seen issues with non-thread safe code.
http://docs.oracle.com/javase/1.5.0/docs/tooldocs/share/jstack.html
Sent from my iPhone
What Java are you running? “jruby -v” will show you both the JRuby
version and underlying Java information.
Not too long ago there was a crash-at-exit issue running JRuby on IBM’s
Java. A recent JRuby change (1.7.1 I think) avoids that specific
problem. There might be other quirks on unusual Java implementations.
Getting a thread dump is a good idea. I find "kill -quit " easier
to use than jstack. The kill will give you a thread dump immediately,
and let the process continue to run. It is great for hung or high CPU
servers. You can get several thread dumps in sequence and see what
changes between them. (“kill” is only available on Unix-like systems,
including OSX and Linux; not available on Windows.)
I don’t expect a “rake db:migrate” to do any problematic threading.
Still, a thread dump will show where it’s stuck.
Thanks for the help here guys. I’ve tried both the kill -quit
option and jstack, and both are very odd.
I first tried the kill -quit to get that stack trace, yet the
process still runs and I’m not seeing any stack trace on stdout (is
there a log of this anywhere you guys can think of? OS X with jruby
installed via rvm).
So next I tried jstack which couldn’t attach, so I tried jstack -F
which failed out:
└─[$] jstack -F 72656
Attaching to process ID 72656, please wait…
attach: task_for_pid(72656) failed (5)
Error attaching to process: Error attaching to process, or no such
process
No such process? I ran ps and sure enough it’s there, it’s running, and
it isn’t zombied. In fact it’s listed as sleeping (S+). Since I’m on OS
X and don’t have strace, I used dtruss to attach to the pid and see what
it’s doing. After letting it run for around ~5-10 minutes, this is all
dtruss is showing me:
└─[$] sudo dtruss -p 72656
Password:
SYSCALL(args) = return
Seriously, one line of output.
I think my next phase is going to be ripping out drip and seeing if that
makes things a bit smoother. I’ll let ya’ll know if that helps.
Thanks again for the suggestions!
Oh and to answer an earlier question:
└─[$] jruby -v
jruby 1.7.2 (1.9.3p327) 2013-01-04 302c706 on Java HotSpot™ 64-Bit
Server VM 1.6.0_37-b06-434-11M3909 [darwin-x86_64]
–
J. Austin Hughey
[email protected]
I noticed hanging happening when I switch between two versions of JRuby
when using Drip.
Hey guys,
Thanks a lot for all your help again. I initially went with the
sledgehammer approach earlier today (been working with this prettymuch
all day) and nuked my entire RVM install, drip, everything, then brought
it all back from scratch. Once that was done, I had a working “normal”,
non-drip jruby going. I replied to a tweet from @headius this morning
(thanks man) saying as much. He stated that if it was drip, he really
wanted to know, so I put drip back in place following directions here:
Under the “Using Drip with an RVM installed JRuby” section. I just
created a new file, copy/pasta’d that code in, chmod +x’d it, and went
off about my business. This is after ‘brew install drip’ and reloading
my shell, by the way.
Then I got the same behavior as before. Some operations would hang just
prior to their normal point of exit. For example, I couldn’t even get
ruby -v (which RVM aliases to jruby) to exit. Same behavior with jstack
and dtruss.
So I removed drip again (deleted that rvm hook, kill -9’d all drip/java
processes, brew uninstall drip, reloaded terminal) and now things are
working normally.
I fully admit that this could be an Error 18 (error is 18 inches from
the screen) but so far the behavior here is quite erratic and I do
confess to being rather confused.
At the moment I’m just going to run dripless (is that even a word?). If
anyone has any thoughts, I’d love to hear it, but so far this looks to
be something funky with drip.
Thanks again all.
–
J. Austin Hughey
[email protected]
Interesting. Perhaps that’s something we should be checking for in
DripMain? Or actually, perhaps Drip should be checking whether the
libraries you’re loading into its classpath have changed?