Maximizing cores with eventmachine / jruby

hello,

Has anyone done any extensive testing with Jruby + Eventmachine to max
out
a large multi-core environment? Will running multiple threads
(somehow?)

  • EM leverage all the cores automatically or is there some tweaking to
    do
    be done to spawn multiple event-loop threads to handle requests ?

We are looking at a Rails/Sinatra + Jruby setup using EM for more async
i/o to get better throughput on some webservices but want to ensure we
can
leverage all 4+ cores in the most efficient way in the event loops.

Cheers
-AD

As far as I know, EM is a single-threaded framework, with a single
event-loop thread and no parceling out of work to worker threads.

You can use something like Celluloid to max out multiple cores, which
is much more thread-oriented. There’s also some interest in building a
light-weight eventing system for JRuby that uses multiple cores but
which does not use fibers, as in Celluloid.

  • Charlie

Shameless plug.

There’s also http://vertx.io/ that uses JRuby and is designed to scale
out seamlessly over available cores

Yeah I don’t know why I keep forgetting about vert.x, Tim!

Would you be interested in doing a guest blog post for blog.jruby.org
about it? I think it needs more recognition and exposure.

Email me off-list if so.

  • Charlie

whats the architectural paradigm behind vertx.io ?

On Thu, Mar 29, 2012 at 3:59 PM, Charles Oliver N.

The manual goes into some detail http://vertx.io/wsver-0002/manual.html

But in a nutshell… one way of thinking about it as a hybrid of the
reactor model (e.g. node.js, ruby event machine) and the actor model
(Erlang, Akka etc). That’s a very loose definition.

With node and event machine you have a single event loop so a single
instance can use at most one core. With vert.x you can have many event
loops in a single JVM instance, so you can scale over available cores,
different event loops can then communicate by messaging passing.

Then, on top of that it’s polyglot, you can use it via Java, Ruby,
JavaScript and Groovy (coming soon), or you can mix and match these
langs in the same project.

On 30/03/2012 21:37, AD wrote:

well that looks really interesting. Been spending a lot more time
with Async I/O stuff and Erlang and this looks really nice. I also
like the use of hazelcast for the event bus.

Is this prod ready? Is anyone running it in prod if so ?

We are currently in beta. Hopefully going 1.0.final in the next month or
two :slight_smile:

well that looks really interesting. Been spending a lot more time with
Async I/O stuff and Erlang and this looks really nice. I also like the
use
of hazelcast for the event bus.

Is this prod ready? Is anyone running it in prod if so ?

I just had to tweet in excitement, ran some tests with a basic Jruby
hello
world and got 7000 req/sec !

Definitely gonna play with this a bit more. Are there any performance
differences between writing the API in Java vs the interpreted
Jruby/RhinoJS ?

-AD

If you run it with -instances you’ll go even higher,
since by default it will use just one core.

Anyway, let’s take this to the vert.x google group, I’m sure not
everyone here wants to hear about vert.x :slight_smile:

On Sat, Mar 31, 2012 at 8:58 AM, AD [email protected] wrote:

Fair enough will do (although i do feel this platform can be a game changer
for Jruby if my initial speed results remain consistent). You basically
have a platform for writing APIs in ruby, providing websockets, async I/O
and a distributed cache based on the actor pattern (I am a huge Erlang fan),
so kudos to you , the platform looks great.

FYI -instances 4 helped,

Request rate: 10479.0 req/s (0.1 ms/req)

Impressive :slight_smile:

  • Charlie

Fair enough will do (although i do feel this platform can be a game
changer
for Jruby if my initial speed results remain consistent). You basically
have a platform for writing APIs in ruby, providing websockets, async
I/O
and a distributed cache based on the actor pattern (I am a huge Erlang
fan), so kudos to you , the platform looks great.

FYI -instances 4 helped,

Request rate: 10479.0 req/s (0.1 ms/req)

:slight_smile:

-AD