JRuby and rspec-rails performance

Ruby JRuby
1

Hello, on my current project, we experimented with JRuby 1.1.1. The
specs passed but the spec suite was over 100% slower (from 19 to 45
seconds).
Is also took over 20 seconds for the rails environment to load (before
the first test was run) while it took less than 10 seconds with MRI.

Charles Nutter claims that JRuby is faster than the MRI, even in
interpreted mode.

What has everybody else’s experiences been regarding JRuby performance
with rspec & rails? Do you think we missed something?
It really seems like something is not right.

Thanks,
Brian

To unsubscribe from this list, please visit:

http://xircles.codehaus.org/manage_email
2

Brian T. wrote:

Hello, on my current project, we experimented with JRuby 1.1.1. The
specs passed but the spec suite was over 100% slower (from 19 to 45
seconds).

For short runs of anything (e.g. < 60s) JRuby usually ends up slower
than MRI. This is partly because the JVM needs some warm-up time, but
also because we don’t start compiling code until it gets hit a lot. The
interpreter changes helped speed things up, but they too need some JVM
warmup time. It’s fairly typical for short runs to be slower than MRI.

Is also took over 20 seconds for the rails environment to load (before
the first test was run) while it took less than 10 seconds with MRI.

Startup should see a marked improvement in 1.1.2. How long does it take
e.h. jruby -e “puts 1” to run for you? That time should be halved in
1.1.2.

Charles Nutter claims that JRuby is faster than the MRI, even in
interpreted mode.
Headius: Promise and Peril for Alternative Ruby Impls

What has everybody else’s experiences been regarding JRuby performance
with rspec & rails? Do you think we missed something?
It really seems like something is not right.

There are definitely some bottlenecks running Rails we have not yet
found, so we’re looking for folks like you to help us narrow it down.
Rails is a complicated framework that really requires an implementation
be fast throughout, so the fact that we’re not markedly faster than MRI
all the time means there’s something we’ve missed.

We’re ready to work with you to make things as fast as possible though
:slight_smile:

  • Charlie

To unsubscribe from this list, please visit:

http://xircles.codehaus.org/manage_email
3

Brian T. wrote:

Thanks, unfortunately I don’t think we’re ready to jump to using JRuby
for this particular project.

The main reason is we would still have the same scalability issues
with JRuby, even though it would provide a performance enhancement.

Maybe you can elaborate on that? What sort of scaling issues? We’ve had
multiple reports of people scaling JRuby on Rails very well…so there
are resources here to help.

  • Charlie

To unsubscribe from this list, please visit:

http://xircles.codehaus.org/manage_email
4

On Sat, May 17, 2008 at 4:21 PM, Charles Oliver N.
[email protected] wrote:

warmup time. It’s fairly typical for short runs to be slower than MRI.
Thanks for your answer. That makes sense.

Is also took over 20 seconds for the rails environment to load (before
the first test was run) while it took less than 10 seconds with MRI.

Startup should see a marked improvement in 1.1.2. How long does it take e.h.
jruby -e “puts 1” to run for you? That time should be halved in 1.1.2.
I think its mainly the time to process the Rails initialization.
Starting small processes, like gem, was reasonably fast.
There are definitely some bottlenecks running Rails we have not yet found,
so we’re looking for folks like you to help us narrow it down. Rails is a
complicated framework that really requires an implementation be fast
throughout, so the fact that we’re not markedly faster than MRI all the time
means there’s something we’ve missed.
One interesting thing about our app is that we are using the desert
plugin.
http://github.com/pivotal/desert

Desert is an Appable Plugins replacement. It redefines Rails’ const
missing handling, in the Dependencies module, to load all files in all
plugins that match the constant.
It also allows plugins to have their own app directory with models,
views, and controllers. It also allows plugins to have plugin
dependencies and gives plugins the oppornutity to override its
parents’ modules.

For example, if User is called for the first time, Desert causes rails
to load user.rb on all of the plugin and application load paths.

This causes more File.expand_path and load to be called than your
typical Rails app. I mention that because its been known to cause
slowness under the MRI in certain situations, and its my initial guess
as to what may be happening.

We’re ready to work with you to make things as fast as possible though :slight_smile:
Thanks, unfortunately I don’t think we’re ready to jump to using JRuby
for this particular project.

The main reason is we would still have the same scalability issues
with JRuby, even though it would provide a performance enhancement.

  • Charlie
    Thanks,
    Brian

To unsubscribe from this list, please visit:

http://xircles.codehaus.org/manage_email

To unsubscribe from this list, please visit:

http://xircles.codehaus.org/manage_email
5

I running off of jruby-trunk on OS X 10.5, and can back up Charlie’s
claim of faster startup times compared to 1.1.1. I’m seeing sub 1
second startup times for most of my ruby code. MRI still spins up in
much faster (0.005s vs 0.500s), but pain point is definitely gone for
me now.

I run quite a bit of non-Rails ruby code, and here’s where I’ve seen
the most benefit in using JRuby over MRI.

  1. Anything net/http related runs much faster.
  2. Much better GC for anything that has to be long running.
  3. Ability to use jconsole to profile.
  4. Using java integration to hook into JMS queues. (Still not a huge
    fan of the stomp and ActiveMQ route.)
  5. Production deployment much simpler when only requirement is the
    JVM. Your system admins won’t hate you. You will get the evil eye if
    you use gems and use a library that has a C extension.

I’ll agree that Rails still isn’t ideal on JRuby yet, but it’s getting
there, and I think the recent release of the JRuby Rack adapter opens
up many more doors. Looking forward to seeing some people do some
tests with scaling Merb.

  • Larry

On May 18, 2008, at 9:02 AM, Christian S. wrote:

Windows (something like 6 secs, JDK6). On Windows JRuby was even
like class-loading/class-redefining is the bottleneck. Requiring
To unsubscribe from this list, please visit:

http://xircles.codehaus.org/manage_email

To unsubscribe from this list, please visit:

http://xircles.codehaus.org/manage_email
6

Larry M.-3 wrote:

  1. Anything net/http related runs much faster.
  2. Much better GC for anything that has to be long running.
  3. Ability to use jconsole to profile.
  4. Using java integration to hook into JMS queues. (Still not a huge
    fan of the stomp and ActiveMQ route.)
  5. Production deployment much simpler when only requirement is the
    JVM. Your system admins won’t hate you. You will get the evil eye if
    you use gems and use a library that has a C extension.
  1. Easy way to share things in memory across Ruby instances (e.g.
    between
    Rails runtimes)
  2. Availability of a variety of great clustering solutions

Larry M.-3 wrote:

I’ll agree that Rails still isn’t ideal on JRuby yet, but it’s getting
there, and I think the recent release of the JRuby Rack adapter opens
up many more doors. Looking forward to seeing some people do some
tests with scaling Merb.

Yes, I think JRuby is great for Rails deployment in production. But I
think
first-time users of JRuby will be irritated by the slow Rails
environment
startup (after having read that JRuby is actually faster), especially in
development mode. And isn’t Rails much about fun at development time? So
I
wouldn’t underestimate this issue.

–
View this message in context:
http://www.nabble.com/JRuby-and-rspec-rails-performance-tp17297210p17308283.html
Sent from the JRuby - User mailing list archive at Nabble.com.

To unsubscribe from this list, please visit:

http://xircles.codehaus.org/manage_email
7

On May 18, 2008, at 5:59 PM, Christian S. wrote:

Yes, I think JRuby is great for Rails deployment in production. But
I think
first-time users of JRuby will be irritated by the slow Rails
environment
startup (after having read that JRuby is actually faster),
especially in
development mode. And isn’t Rails much about fun at development
time? So I
wouldn’t underestimate this issue.

On another note. I’m coming over from the camp of traditional java
servlet type apps and compared to there, Jruby/Rails is a treat(IMO).
Starting my Rails server might take a few seconds(Not noticeably
longer than it used to take Tomcat to start or reload my app in
Tomcat), but after that I can continue rarely having to do it again.
The change/feedback cycle is worlds better than what I used to deal
with.

-lenny

To unsubscribe from this list, please visit:

http://xircles.codehaus.org/manage_email
8

Lenny M. wrote:

tests with scaling Merb.
wouldn’t underestimate this issue.

Came from Java, too… Yes, this is really nice to work with and I agree
once webrick has started up JRuby performance isn’t much of an issue.
But
it’s about the test-cases, which span everything from scratch everytime.

And of course it’s often the first impression that counts, that’s why
it’s
probably hard right now to convince MRI users that JRuby is actually the
better Ruby VM.

–
View this message in context:
http://www.nabble.com/JRuby-and-rspec-rails-performance-tp17297210p17323347.html
Sent from the JRuby - User mailing list archive at Nabble.com.

To unsubscribe from this list, please visit:

http://xircles.codehaus.org/manage_email
9

Brian T. wrote:

Hello, on my current project, we experimented with JRuby 1.1.1. The
specs passed but the spec suite was over 100% slower (from 19 to 45
seconds).
Is also took over 20 seconds for the rails environment to load (before
the first test was run) while it took less than 10 seconds with MRI.

I’m experiencing very similar numbers, loading the Rails environment is
really slow with JRuby: 12sec compared to 1.7sec with MRI.

Number are slightly better when I switch to client Hotspot (jruby
-J-client): 9sec

I moved to Ubuntu recently and I remember a better JRuby performance on
Windows (something like 6 secs, JDK6). On Windows JRuby was even
slightly
faster than MRI.

With JRuby being slower on Linux while MRI being much faster on Linux
(compared to Windows), JRuby is really a deployment-only option for me
right
now.

I did some very basic profiling of the Rails startup procedure and it
seems
like class-loading/class-redefining is the bottleneck. Requiring
active_support alone takes almost 6 seconds (e.g. the core extensions
account for 3 seconds).

–
View this message in context:
http://www.nabble.com/JRuby-and-rspec-rails-performance-tp17297210p17302698.html
Sent from the JRuby - User mailing list archive at Nabble.com.

To unsubscribe from this list, please visit:

http://xircles.codehaus.org/manage_email
10

On May 19, 2008, at 1:01 PM, Christian S. wrote:

getting
startup (after having read that JRuby is actually faster),
Tomcat), but after that I can continue rarely having to do it again.
everytime.

And of course it’s often the first impression that counts, that’s
why it’s
probably hard right now to convince MRI users that JRuby is
actually the
better Ruby VM.

Agreed on that. The startup time is definitely an issue in the
context of testing. I’ve actually decoupled much of my code from
Rails so I can test/spec without having to load the rails environment
which as someone else mentioned swallows more than a few seconds.
I’ve been meaning to look into spec_server as a possible remedy to
this, but I couldn’t get it working easily the last time I tried.

To unsubscribe from this list, please visit:

http://xircles.codehaus.org/manage_email
11

Lenny M. wrote:

Agreed on that. The startup time is definitely an issue in the
context of testing. I’ve actually decoupled much of my code from
Rails so I can test/spec without having to load the rails environment
which as someone else mentioned swallows more than a few seconds.

I was extremely annoyed with this and ended up with modifying my Rails
application so that it could run tests both under JRuby and Ruby (MRI).
A few

if RUBY_PLATFORM =~ /java/
…
end

did the trick. I added some more information on the URL below for
anyone that is interested.

http://www.norlin.se/blog/2008/12/09/40/

Cheers,
Stefan

View this message in context:
http://www.nabble.com/JRuby-and-rspec-rails-performance-tp17297210p20922250.html
Sent from the JRuby - User mailing list archive at Nabble.com.

To unsubscribe from this list, please visit:

http://xircles.codehaus.org/manage_email