Zed and Luis drop the bomb on Ruby's poor performance

Ruby
1

The O’Reilly blog has two great interviews posted from the last week
or so. The first [1] is with Zed S., the author of Mongrel (and a
bunch of other Ruby frameworks and utilities). A follow-up interview
with one of the contributors on the Mongrel project, Luis L. [2],
covered some similar ground.

What I found most striking about both interviews was their mention of
YARV and Ruby’s poor computational performance. Zed said, and I quote:

I?ll be honest right away though and say that Ruby is slow. The
Ruby community has been ignoring the huge ?performance? elephant
standing in the room and they need to start talking about it so it
goes away. Elephants hate being talked about. There are a few
efforts to make Ruby faster, but I see a lot less action than is
needed to solve the problem. One solution in the works is a real
virtual machine called Rite (or YARV depending on who you talk to)
which is showing some real promise and seems to be speed
competitive with the fastest Java implementations.

I tend to agree with this. Whenever the subject of computational
performance comes up, quite a few posts end up in the thread
questioning the meaning of the word performance and turn it into an
argument over semantics. Is it coding speed, clarity of thought,
readability, etc. To me that kind of crazy semantic dodge doesn’t
serve us well. Those discussions should be remain on computational
performance since the language has been (mostly) stable in syntax for
years.

So what is being done to get YARV the attention it deserves? Why
aren’t we talking about it all the time? What can I personally do to
help? Are there valid excuses for not focusing on improving Ruby’s
poor computational performance?

cr

[1] O'Reilly Media - Technology and Business Training
[2] O'Reilly Media - Technology and Business Training
interview_with_luis_lavena_2.html

2

One question I have to ask is “Wouldn’t it make more sense to piggy back
on parrot (the Perl v6) engine?” and get all the benefit of the
performance improvements that parrot will pick up and the fact that
there are many, many very clever people working on parrot whose work we
could make use of.

Is there a reason why we are reinventing the wheel?

3

unknown wrote:

So what is being done to get YARV the attention it deserves?
AFAIK YARV will be the default VM when ruby 2.0 comes out. And I think
2.0 will be the next release of ruby. Becoming the default VM seems like
as much attention as you can get.

Why aren’t we talking about it all the time?
It will be in the next major release, what more is there to talk about?
:slight_smile: At the moment it is still not 100% compatible with ruby 1.8 (and
might never be), but it works for a lot of ruby scripts/programs.

What can I personally do to help?
Check out their site:
YARV: Yet Another Ruby VM

Edwin

4

snip

So what is being done to get YARV the attention it deserves? Why
aren’t we talking about it all the time? What can I personally do to
help? Are there valid excuses for not focusing on improving Ruby’s
poor computational performance?

There vertainly are perfectly good “excuses” (I didn’t know I needed
one! :slight_smile: ) for not focussing on Ruby’s performance - it’s performance is
only a concern for one (poor or otherwise) if it’s actually causing one
a problem.

In my previous incarnation as a signal processing and volume rendering
coder, I imagine Ruby’s performance would have been unnaceptable. In my
current role as a testing framework programmer, it’s plenty fast thanks,
and jolly expressive. I’ve no reason (at the moment) to require more
performance from it. When I switch back to doing graphics again, I’ll be
generating my c, c++ or assembly with Ruby anyway, rather than hand
coding it, and it’ll probably be fast enough for that too.

All that said - I’ll crack open a drink to Ruby’s performance improving.
I just want to point out why it doesn’t (pragmatically) matter to me
personally atm.

Cheers,
Benjohn

5

On 5/22/06, Peter H. [email protected] wrote:

One question I have to ask is “Wouldn’t it make more sense to piggy back
on parrot (the Perl v6) engine?” and get all the benefit of the
performance improvements that parrot will pick up and the fact that
there are many, many very clever people working on parrot whose work we
could make use of.

Is there a reason why we are reinventing the wheel?

http://wiki.rubygarden.org/Ruby/page/show/VirtualMachineOptions

Cardinal
A re-implementation of Ruby on top of the [Parrot] virtual machine
This is the virtual machine that’s being developed to support Perl 6
and other languages.

Availability
Parrot is currently in development. The Cardinal project has it’s
homepage at [RubyForge]. However it has been dormant since August
2004.

In [languages/ruby] of the Parrot SVN repository there is an earlier
attempt at targetting Ruby on Parrot. The last real update of that was
in August 2002.

Advantages

  1. Access to large Perl development community and resources
  2. Will be able to use modules developed in and/or for Perl and
    other Parrot-based languages
  3. The Ruby interpreter could be written in Ruby
  4. Native threads
  5. Precompiled bytecode
  6. JIT
  7. Possible speed improvement (but note that Parrot and Cardinal
    are still in development)

Disadvantages

  1. Different C interface (But Parrot’s C interface is also clean)
  2. Relies on Parrot, a large work-in-progress

Notes

  1. Not an “official” VM; when asked about VM Matz himself answered
    he prefers to develop his own VM
6

On 5/22/06, [email protected] [email protected] wrote:

I just want to point out why it doesn’t (pragmatically) matter to me
personally atm.

Unfortunately, not all of us are in your boat. I really like Ruby, but
if performance improvement is not going to be in Ruby’s near future,
my management is not going to buy into Ruby!

In most of the corporate environments I have worked in, many managers
tend to take IT decisions based on ‘feelings’; ‘speed’ is a word that
creates good ‘feelings’. So, when the issue of performance is bought
up, Ruby doesn’t win (doesn’t matter if the application requires it or
not).

IMO, if Ruby’s vision is to penetrate the corporate environment, Ruby
should address its performance issues among other things (e.g. support
for apps – Ruby on Rails).

-Madan.

7

Madan, can you elaborate on this:
“(e.g. support for apps – Ruby on Rails)”

thanks

8

On 5/22/06, Madan M. [email protected] wrote:

IMO, if Ruby’s vision is to penetrate the corporate environment, Ruby
should address its performance issues among other things (e.g. support
for apps – Ruby on Rails).

I don’t think it is Ruby’s (or Matz’s) vision for Ruby to penetrate
the corporate environment. Keep in mind that unlike Java or C#, Ruby
is not a commercial language, and there isn’t really any benefit to
massive corporate uptake of Ruby. For those of us that enjoy the
language it certainly would be nice for our choice to be “validated”
by getting more market share, but in general myself and probably most
other Rubyists don’t put much value on corporate use being an
indicator of language popularity.

Still, performance is important, and YARV will be a great improvement
in that area.

Ryan

9

Pretty much the same for me. If Ruby is not fast enough then I convert
it to C, nothing less than C seems to have the performance if
performance is an issue (ok in extreme cases there is assembler but
nothing I have ever written needed to be written in assembler -
besides it’s not as much fun since the m68k went south). The pain of
going from Ruby to Java is not going to realise the performance
improvement of going from Ruby to C for, what appears to me to be, not
quite as much pain.

10

On Monday 22 May 2006 9:25 am, Madan M. wrote:

On 5/22/06, [email protected] [email protected] wrote:

I just want to point out why it doesn’t (pragmatically) matter to me
personally atm.

Unfortunately, not all of us are in your boat. I really like Ruby, but
if performance improvement is not going to be in Ruby’s near future,
my management is not going to buy into Ruby!

The thing that always stands out to me in these performance discussions
is
that people argue that Ruby needs a performance improvement, but there
are
rarely any specifics attached to that analysis.

How much of speed improvement? How fast should Ruby be? For what?

I like speed in my applications. It gives me a warm, fuzzy feeling.
However
Ruby’s speed has never been a problem for me, and I have been using Ruby
in
enterprise applications, for at least some definitions of enterprise,
for 4
years.

On really very modest hardware it is no problem for me to dynamically
generate
web pages for sites/applications fast enough that it could handle
millions
of hits a day. That is, if I can optimize away the cost of most
database
interactions through caching or other mechanisms. Ruby could be 10x
faster,
and if I have to make database queries with each request, it’s not going
to
matter, because Ruby isn’t the bottleneck, most of the time.

I do a lot of data/file generation/handling/transformation/transport
duties
with Ruby, and Ruby is never the bottleneck.

I do email build/send tasks with Ruby, and Ruby isn’t the bottleneck.
In
fact, I have a number of years experience in the dynamic, targetted
email
business, and my relatively simple Ruby based email build platform is
more
powerful, easier to use, and faster than the quite expensive (in
developer
hours spent to build/optimize it) Java based system that my last
employer
used.

Ruby can be a bottleneck, but for the vast majority of Ruby users and
the vast
majority of tasks people use if for, it is not the bottleneck.

It is not an elephant in the living room that everyone is ignoring if
the
performance simply isn’t a limiting factor. It’s an elephant out in the
paddock, where it belongs in those cases. And those cases are the
majority
of cases.

Kirk H.

11

On May 22, 2006, at 11:25 AM, Madan M. wrote:

In most of the corporate environments I have worked in, many managers
tend to take IT decisions based on ‘feelings’; ‘speed’ is a word that
creates good ‘feelings’. So, when the issue of performance is bought
up, Ruby doesn’t win (doesn’t matter if the application requires it or
not).

Personally, I like it just fine when companies make decisions based on
feelings rather than logic. This is just one reason why there is
always room in the market for a new competitor.

Gary W.

12

The problem is that Zed and Luis make out that performance is something
that the whole Ruby community is deliberately ignoring. This is not the
case, there are plenty of people who have no issue with the performance
of Ruby. The truth is the performance is an issue for Zed and Luis and
they are trying to make out that it is not their personal bugbear but
some sort of conspiracy so they don’t look like whiners. I have never
though “damn Ruby’s dismal performance, it should be able to do this
faster” and I probably never will.

Yes we wouldn’t say no to better performance but we (the Ruby community)
are not being held back by the current level of performance.

13

On May 22, 2006, at 10:07 AM, [email protected] wrote:

one
generating my c, c++ or assembly with Ruby anyway, rather than hand
coding it, and it’ll probably be fast enough for that too.

Not to single you out (though I guess I am) but you are the first one
in the thread to write-in with “it’s good enough for me because of
.”

I think we all know this. Most of us on this list are here because we
find the language expressive, clean, fast enough for whatever, yada
yada yada.

This thread is specifically about getting more computational
performance for Ruby. Let’s not throw it off track by reiterating all
the other reasons the language is “good enough” for us.

All that said - I’ll crack open a drink to Ruby’s performance
improving.

Good! Me too!

I just want to point out why it doesn’t (pragmatically) matter to me
personally atm.

Oops! Wrong thread! :slight_smile:

cr

14

On Tue, 23 May 2006 [email protected] wrote:

All that said - I’ll crack open a drink to Ruby’s performance improving.

Good! Me too!

I just want to point out why it doesn’t (pragmatically) matter to me
personally atm.

Oops! Wrong thread! :slight_smile:

cr

you make good points - but let me just throw in my 2 cts:

virtual machines are stupid. java is slow. c++ is slow. python is
slow.
ruby is slow. so what. if you want fast you need c or fortan or
assembler.
all three are very easy to call from ruby. i use mmap, narray, gsl,
ruby/dl,
ruby queue to cluster and other combos to process GB sized images on a
routine
basis. people are doing real-time video processing with ruby using a
similar
approach. the key to speed is c, not a vm and, if you ask me, that
quest is
the white elephant - not ruby’s speed. projects like ruby2c,
ruby-inline,
ruby/dl, swig - those are the way to speed. putting a vm on top of ruby
and
trying to make it fast is like chopping and dropping your mini-van and
heading
to the track : you are going to get killed. i think it’s right to
demand a
specfic need for speed because, if people do, i think it will become
apparent
that any vm will have a hard time competing with a little thinking, a
lot of
ruby, and a little c via some of the nice projects that let us it. and,
if we
stick to gcc, we also get the best portability of any code generator out
there. lastly, this approach is here now - we don’t have to wait.

just my thoughts.

-a

15

This thread is specifically about getting more computational
performance for Ruby. Let’s not throw it off track by reiterating all
the other reasons the language is “good enough” for us.

How much more? Ruby will never be as fast as hand written assembly
code, well written C, etc. The problem with the performance debate is
premature optimization. When things are too slow, optimize the 5% of
your code that is too slow (after careful profiling to locate that 5%)
– if necessary as a C extension.

I am all for a fast Ruby, YARV will likely deliver when it is ready.
Even with YARV and a some future JIT compiler, there will be times
when your application is too slow and will require optimization.
Making Ruby faster is not going to help in many cases, you still need
to profile and optimize.

Can Ruby’s performance be increased? Yes, but the only “elephant” in
the room is the astral projection of Sir Charles Antony Richard Hoare
muttering that the root of all evil is premature optimization…

pth

16

Hi cr,

You stated that this kind of comment is off the thread topic:

I just want to point out why it doesn’t (pragmatically) matter to me
personally atm.

Oops! Wrong thread! :slight_smile:

cr

But when you started the thread one of the questions you asked was:

“Why aren’t we talking about it all the time?”

“it” being the performance issue. If this was not one of the topics you
wished to discuss in this thread it would be better not to specifically
ask about it.

I think we aren’t talking about it all the time because most of us don’t
have the expertise to contribute to the design and programming of a Ruby
byte-code compiler however I am closely following the progress of YARV.

Here’s a concrete example of something I am doing in Ruby that I would
like to be faster:

I am using REXML and ruby-spreadsheet to read XML archives of Blackboard
courses (an e-learning platform) and to process the forum discussions to
produce interesting (at least to the education researchers I work with)
statistics of all kinds about the conversations learners have in our
on-line courses.

Right now it takes about 30s to process a BB archive and produce the
excel report. I am VERY happy programming using the native ruby tools
for parsing the BB XML and writing my object representations back out as
excel worksheets. These libraries are almost trivial to acquire (gem for
the spreadsheet) and to use. However if I was to instead make an
interactive application I’d like the process to be sped up by about 10x.
I could use the ruby bindings for the C libxml library if I need to but
I’d rather stay totally in Ruby. Until Ruby 2.0 is available I am quite
happy that if I need the performance I can use a C library to speed up
the slow parts.

Something practical: I’ll test my relatively simple app with the latest
YARV and post here and to the YARV folks my results.

FYI: Downloading and installing YARV: YARV: Yet Another Ruby VM

17

Kirk H. wrote:

The thing that always stands out to me in these performance discussions is
that people argue that Ruby needs a performance improvement, but there are
rarely any specifics attached to that analysis.

Very good point.

…

Ruby can be a bottleneck, but for the vast majority of Ruby users and the vast
majority of tasks people use if for, it is not the bottleneck.

It is not an elephant in the living room that everyone is ignoring if the
performance simply isn’t a limiting factor. It’s an elephant out in the
paddock, where it belongs in those cases. And those cases are the majority
of cases.

Or it’s the hamster in the living room. People don’t comment on it
because, for the most part, they don’t see it.

Those people having speed issues: Is it not a reasonable option to
replace the bottleneck Ruby code with custom compiled C libraries?

I tend to offer that as the general speed solution, but don’t have all
that much experience with it. I’ve written Ruby modules in C as an
educational exercise, and found it to be as easy as people say it is,
but I’ve not had to do it in Real Life.

I’m curious if this is as good an option as I’ve been inclined to
believe.

–
James B.

“You harmonize; then you customize.”

  • Wilson Pickett
18

Peter H. wrote:

[…] The truth is the performance is an issue for Zed and Luis and
they are trying to make out that it is not their personal bugbear but
some sort of conspiracy so they don’t look like whiners. […]

Nice comment Peter!

My comments aren’t “look, I cannot code a good program… blame ruby for
its poor performance”

Ruby do his job, and do it well (as I said in the interview)

I work with special MPEG2 playback hardware, some of them analog only,
others with digital interface (ASI-DVB)

Even that I like ruby, cannot use to interface that hardware, mostly
because the quantity of computation needed in the results turns ruby in
a slow, memory comsuming pig.

I don’t say that YARV and later a JIT compiler will solve this, but will
provide better performance in general.

I came from a Python background, and ruby was “slow” compared to it
(http://shootout.alioth.debian.org/gp4sandbox/benchmark.php?test=all&lang=ruby&lang2=python)

But I when come to programing speed (or performance) Ruby wins.

Yes we wouldn’t say no to better performance but we (the Ruby community)
are not being held back by the current level of performance.

As another community member, I agree with you… thats why I keep
working in Ruby :slight_smile:

Later,

Luis

19

From: [email protected] [mailto:[email protected]]

similar
approach. the key to speed is c, not a vm and, if you ask me, that quest
is
the white elephant - not ruby’s speed. projects like ruby2c, ruby-inline,
ruby/dl, swig - those are the way to speed.

Makes sense. But.
I’d like to think about things like “reasonable slow” and “unreasonable
slow”; and Ruby sometimes is just unreasonable slow. Here are some
examples:

  • I’m working with custom UI library, which is based on HTML (namely, on
    HTMLayout – Terra Informatica Software). There are a large amount of objects
    created
    corresponding to DOM tree nodes (“large” means thousands) and profiler
    says,
    that one of the slowest operation is Class#new - is it normal?

  • 5000 objects is loaded from database (by SELECT’ing and setting
    fields).
    The major slower is that each object has Date field (Integer#gcd is very
    slow). Or Kernel#send (I use it because of my DSL created) and so on.

Summarizing, after large profilings and optimizations, there are oftenly
some core methods (even Fixnum#+) show themselves as major speed
problems.

This situation I call “unreasonable slow”.

-a

Victor.

20

On 5/22/06, [email protected] [email protected] wrote:

The O’Reilly blog has two great interviews posted from the last week
or so. The first [1] is with Zed S., the author of Mongrel (and a
bunch of other Ruby frameworks and utilities). A follow-up interview
with one of the contributors on the Mongrel project, Luis L. [2],
covered some similar ground.

First, thanks for reading and pointing out the interviews. I’m glad
people
are enjoying them. I’m hoping to do some more in the coming
weeks/months.

virtual machine called Rite (or YARV depending on who you talk to)
which is showing some real promise and seems to be speed
competitive with the fastest Java implementations.

I think you’re quoting selectively here. There are some counter points
in
both interviews. Luis said:
“For us, Ruby on Rails (RoR) lets us build fast, clean
(especially clean) prototypes within hours or days, not
weeks or months. Then we can move from prototype to
final product within weeks, even for big projects (like
migrating a whole application from Zope to RoR).”
(emphasis added)

Zed said:
“Ruby’s advantage though is not in it’s blazing
execution speed but it’s blazing coding speed.
I’ll put it to you this way: I wrote mongrel in about
3 months. That’s a full featured stable web server
that can run four Ruby web application frameworks
and is already powering many Ruby web sites. This
wouldn’t be possible without Ruby the language.”
He also wrote:
“[A] Rails application many times can outperform
similar applications in Java or PHP.”

This doesn’t mean Ruby is a speed demon, or that Zed and Luis
don’t want to see general speed increases (we all do, and they
seem to be on the horizon).

For most users (as has been repeated on this thread a few times),
Ruby is fast enough, and easy to optimize when it’s not (RubyInline
for pure Ruby, cacheing for Rails).

I tend to agree with this. Whenever the subject of computational
performance comes up, quite a few posts end up in the thread
questioning the meaning of the word performance and turn it into an
argument over semantics. Is it coding speed, clarity of thought,
readability, etc. To me that kind of crazy semantic dodge doesn’t
serve us well. Those discussions should be remain on computational
performance since the language has been (mostly) stable in syntax for
years.

I think the main argument you will run into (over and over again) is,
have
you profiled your code? Do you know what’s running slowly? If not, you
may just be moving through a bottleneck a little faster once YARV gets
here. Really, I’m all for improvements to Ruby in general, but
profiling
and optimizing your bottlenecks by writing better code will help you out
of a number of tight spots.

So what is being done to get YARV the attention it deserves? Why
aren’t we talking about it all the time? What can I personally do to
help? Are there valid excuses for not focusing on improving Ruby’s
poor computational performance?

Are you on the Ruby Core mailing list? Are you posting bugs and
feature requests to the trackers? Have you downloaded and built
YARV so that you can test it on your system? All of these are ways
you can get involved and help out.