JRuby performance

Ruby JRuby
1

I just found out about Web Framework Benchmark at

I found it interesting to find a lot of their JRuby benchmarks performed
worse than MRI one. I read their test methodology, they do allow some
warm
up period to make sure JIT based platform has enough time to kick in.

Any thought why JRuby performs worse, when supposedly better than MRI
one?

2

Had a look at their github repo… Gemfile for jruby contains
jruby-jars…
So it might be jruby running within MRI, not jruby running in pure java.

So far over more than 2 years moving from MRI to jruby, I never found a
case where jruby would perform so badly, and I am on Windows…

Using a fully expanded or installed jruby is faster than jruby-jars or
jruby-complete, especially with lots of gems.

3

We only perform worse on the rack benchmarks for all others we perform
better. Perhaps there is something in jruby-rack which is not
performing
well? I would think if a full Rails app warms up enough to perform
better
than MRI a smaller app would warmup faster?

-Tom

4

It’s likely that the tests aren’t CPU-bound. These tiny Rack apps are
basically a test of how much the web server, language, and framework can
get out of the way. And my guess is there’s either something in
JRuby-Rack or Resin that is getting in the way - excessive object
allocation, synchronization in the request path, etc.

Ben

5

I’ve actually been looking at these benchmarks the past week in
prototyping the next generation of TorqueBox. So far I’ve been looking
mainly at the JSON serialization test and the plaintext test to get a
benchmark.

All the JRuby tests are run on top of Resin. Why Resin was chosen
instead of Puma, Trinidad, TorqueBox / torquebox-lite, or something else
more sane I’m not sure. That’s the biggest problem with at least the
JSON and plaintext tests.

Techempower gets 69k req/s on their i7 hardware for the ‘rack-ruby’ JSON
test that uses Ruby 2.0, Unicorn, and Nginx.

As a baseline, on my local laptop I get 32k req/s for that same
rack-ruby test with the same versions of MRI Ruby, Unicorn, and Nginx.

On that same local laptop with torquebox-lite, I get 46k req/s. A nice
improvement for sure, but not anything amazing.

Still on the same laptop, using some unpushed prototype code that will
replace torquebox-lite, I get 71k req/s. This prototype is based on
JBoss Undertow, which you’ll notice is on top or near the top of most of
these benchmarks. Perhaps it’s time to put a bow on this server and get
it out there to let people play with?

So, we should definitely work up a pull request to use a better JRuby
server option. And it doesn’t have to be TorqueBox by any means - these
are just the simple examples I had locally.

One optimization in all my local JRuby tests is I set the JRuby option
“-Xjruby.ji.objectProxyCache=false” which will be the default in JRuby
9k. At lower throughput levels it doesn’t really make a difference but
once you start getting over 50k req/s or so the overhead from using the
object proxy cache really starts to show up.

Ben

6

These particular tests weren’t using any templating - just strings
returned directly from Rack applications.

7

If anyone else is interested, I’ve released version 0.1.0 of the
‘torqbox’ gem, which scores drastically higher on the TechEmpower JSON
and plaintext benchmarks on my machine. I’d like to get a couple more
reports of its performance before submitting a pull request to the
TechEmpower repo to replace its usage of Resin with torqbox. This should
outperform any other JRuby web server, and I’d like to hear reports from
anyone that can find an example showing otherwise.

If you’re curious, ‘torqbox’ is the codename for a future TorqueBox
version. The funky gem name is just so we can cut releases of both
without confusing current ‘torquebox’ gem users.

Read more at GitHub - torquebox/torquebox at 5x-dev

8

As for the JRuby-Rack part … yes there might be some “redundant” (as
Tom
mentioned) work in JRuby-Rack’s Request/Response lifecycle.

It was not an issue (at least for now) - it simply outperformed MRI as
much
as I’ve seen with Trinidad and some of the standalone servers - Warble-d
(including JBoss).
I have started some work on this locally … e.g. rewriting
JRuby::Rack::Response to “plain” Java, but time is a precious commodity
managing a few OSS projects :slight_smile:

So we’ll see, hopefully some of it gets out this year … at least in a
1.2.0 pre-release.

The other thing is, I’ve heard quite a lot of “praise buzz” about Resin
but
it’s somehow one of the few servers I never actually seen at work even
through my Java days.

K.

9

I wonder also what templating system they were using for rendering
responses. When I first started using JRuby, my existing Rails apps were
10
times slower using JRuby 6, and I couldn’t understand why, but I finally
narrowed it down to anything using haml being terrible (and I was using
a
lot of haml). Erb templates rendered much faster, but still way slower
than
MRI. JRuby 7 got haml down to only 2 or 3 times slower, and Erb was very
close, but like these benchmarks, still a little slower. This makes
decent
sense. It’s really hard to see how Java is going to be faster at
something
like String manipulation than C. But unless an application is really
template/view heavy, I think you’ll make it up in other ways.

Sent from my mobile device. Apologies for brevity.

On Nov 13, 2013, at 3:51 PM, DJ [email protected] wrote:

I just found out about Web Framework Benchmark at

I found it interesting to find a lot of their JRuby benchmarks performed
worse than MRI one. I read their test methodology, they do allow some
warm
up period to make sure JIT based platform has enough time to kick in.

Any thought why JRuby performs worse, when supposedly better than MRI
one?