Zed and Luis drop the bomb on Ruby's poor performance

Ruby
41

Quoting [email protected], on Tue, May 23, 2006 at 05:36:28AM +0900:

Don’t forget that string comparison is pretty quick if the strings
differ in the first character. If you try comparing to the same
string and make the string a lot longer (say 2000 characters) you’ll
get a situation where symbol comparisons are about three times faster
than string comparisons.

But the strings in my app are short, about 20 chars.

Have you tried profiling your code?

Yes, performance was dominated by string comparisons, I wrote the
benchmark to see if changing to symbols would help.

Sam

42

It is not an elephant in the living room that everyone is ignoring if the
performance simply isn’t a limiting factor. It’s an elephant out in the
paddock, where it belongs in those cases. And those cases are the majority
of cases.

Or it’s the hamster in the living room. People don’t comment on it
because, for the most part, they don’t see it.

No! It’s the hamster in the circus! Or no! It’s the elephant in the
garage! It’s the midget on top of your car! It’s the rabbit in the
moon!

Sorry, I’m having issues taking this seriously…I saw the subject
header and I was like, “Somebody set us up the bomb!”

43

On Tue, 23 May 2006, Bob H. wrote:

Ruby’s focus is on something other than performance.

exactly. write poetry in french - scream insults at hockey games in
english.

Not for a lot of the examples that come up.

Definitely not for me.

my stance exactly.

nice summary btw.

-a

44

[email protected] wrote:

On Tue, 23 May 2006, Bob H. wrote:

Ruby’s focus is on something other than performance.

exactly. write poetry in french - scream insults at hockey games in
english.

Unless they’re French-Canadian.

–
James B.

“Programs must be written for people to read, and only incidentally
for machines to execute.”

  • H. Abelson and G. Sussman
    (in "The Structure and Interpretation of Computer Programs)
45

On 5/22/06, Peter H. [email protected] wrote:

One question I have to ask is “Wouldn’t it make more sense to piggy back
on parrot (the Perl v6) engine?” and get all the benefit of the
performance improvements that parrot will pick up and the fact that
there are many, many very clever people working on parrot whose work we
could make use of.

Are there? Parrot has always seemed like a dead Parrot, to me, with
far too much trying to be done in too poorly a way.

Is there a reason why we are reinventing the wheel?

Yes. Ask ko1 and matz why for specifics, but essentially Ruby is more
dynamic than other VMs expect it to be.

-austin

46

On May 22, 2006, at 9:32 PM, James B. wrote:

[email protected] wrote:

On Tue, 23 May 2006, Bob H. wrote:

Ruby’s focus is on something other than performance.
exactly. write poetry in french - scream insults at hockey games
in english.

Unless they’re French-Canadian.

Hmmm? French-Canadians are pretty good at poetry… Oh, you mean the
insults… well of course it’s in english… the english in Canada
don’t speak french… not a lot of fun if nobody understands your
insults :slight_smile:

Cheers,
Bob

Bob H. – blogs at <http://www.recursive.ca/
hutch/>
Recursive Design Inc. – http://www.recursive.ca/
Raconteur – http://www.raconteur.info/
xampl for Ruby – http://rubyforge.org/projects/xampl/

47

On 5/22/06, Madan M. [email protected] wrote:

On 5/22/06, pat eyler [email protected] wrote:

On 5/22/06, Madan M. [email protected] wrote:

On 5/22/06, Kirk H. [email protected] wrote:
Read what Zed said about Rails app performance though. Even though
Ruby can be slow, he said that with its built in caching, Rails apps are often
faster than Java Web apps.
Pat, that is exactly the point: Ruby is slow; so you have to create
work arounds! Not something I would like to hear, if I were evaluating
a language.

No, that’s not the point. That’s you missing the point. Caching is
usually better than recomputing for certain classes of applications in
any case, even if it’s partial caching. There’s a reason that you use
templates wherever possible in output – it’s partial caching.

We need a version of Ruby that is comparable in execution speed to
Python or Java. If it is YARV, then let us roll it out fast (no pun
intended)

It will be rolled out in its time.

And, to be honest, Ruby is comparable to the others in execution.
Numbers, Madan, numbers. Hard numbers will run every time, not
consultant feelings.

-austin

48

On 5/22/06, Madan M. [email protected] wrote:

performance issues with Ruby (and YARV will solve it in the future).
How am I now to convince my manager that Ruby is ‘better’ (within
the context of performance)?

Zed is but one of many “authorities” on Ruby. He’s got certain
experiences, but others – and there’s a lot of them – have different
experiences.

Why should we develop your benchmarks to challenge something that
consulting companies are lying about? If someone gives you benchmark
results without publishing the source code for review and having
independent verification of those results with properly tuned algorithms
in both cases – they are lying. It’s sort of like the Alioth shootout
(that someone has already posted about). There’s not a lick of worthhile
truth on that site, because there are implementations of algorithms that
are unmaintainable (and other implementations that require esoteric
hacks to get working).

The reason why IT managers want ‘performance’ even if the application
really doesn’t require it, is to cover their behinds, among other
thing. For example, if the website is slow, they don’t want it to be
attributed to the language, even if that is not the case (could be a
network problem, but the network group would more willingly point
their fingers at the app. development group for their choice of
language than to accept their problem); has happened before and will
happen again.

Then you need to have stronger IT managers. The development group should
be familiar with various profiling tools if they have an asinine network
group who points fingers without absolute proof.

Sorry, but it’s true. Performance of a website/application is the
responsibility of the entire company.

So, here ‘performance’ is not for the sake of performance demanded by
the application, but for the sake of ‘not wanting to be talked to by
the boss’. That is the reason I said that many IT managers make
‘performance’ decisions based on ‘feelings’, not necessarily had data.

It is usually bad data, though.

(2) Using two different languages for different performance needs:
Some fellow Rubyists have said that they would switch to C if their
app. needed performance boosts. This will work if it is a personal or
small project. But when in an large corporation, switching between
languages to get performance gains will result in maintenance
nightmares. Many IT managers understand this and have experienced
this. Combined that with the mantra ‘Java is fast’ preached by many
consultants, and the decision for the senior IT manager becomes easy:
one language that can be used for apps that require real performance
and for ones that don’t: Java.

This is not correct. I have worked in several companies and have used,
over the years, no fewer than fifteen languages. We use the language
that is most appropriate to the task at hand. Now, switching between
Java and C is a maintenance nightmare because JNI sucks so badly.

(Remember: those same consultants who are now preaching “Java is fast,
Ruby is slow” were the jackasses a decade ago who were preaching “C++ is
fast, Java is slow”.)

(3) Survival of Language: This is another bigee. If the IT manager OKs
a language that happens to be a fad, then he might as well bid goodbye
to his job (lost productivity, cost, etc). So, they want to be sure
that the language is going to be in the ‘main stream’ for a long time.
In their mind, the only way they can be assured of that is:

This is the least worry. Ruby will survive with or without “enterprise”
support.

A good case study would be on how Linux got its foot hold into the
corporate world.

Usually? Through the back door.

Once again, if Ruby’s vision is not to penetrate the corporate
environment as Ryan pointed out (but would be a nice validation if it
indeed penetrate the corporate world by some means), then this
discussion is moot. Knowing that, I wouldn’t try to push Ruby to my
managers; I will use it for personal projects.

Then that would be your mistake. Know the battles you can fight. Prove
Ruby useful and its use will grow.

On the other hand, if Ruby has a dream of penetrating the corporate
world, then it needs a marketing machine that would deal with the real
and, more importantly, the ‘illusionary’ problems created in the
corporate world – just like Firefox’s or Linux’s marketing machine.
Unfortunately, penetrating the corporate world by some means is
never going to happen, unless an concerted effort is made.

Fortunately, Ruby doesn’t need this.

-austin

49

On Tue, May 23, 2006 at 03:39:37AM +0900, Madan M. wrote:

work arounds! Not something I would like to hear, if I were evaluating
a language.

That is exactly the other point. Rails has superior built in
caching that makes up for any perceived or true interpreter slowness.
Caching is not a workaround, it is the solution to a problem, already
solved for you by the rails team, and easily reuseable.

If said web apps are faster than J2EE already, will your manager care
if the interpreter itself is slow? Will he even understand?

Jürgen

50

Peter H. wrote:

One question I have to ask is “Wouldn’t it make more sense to piggy back
on parrot (the Perl v6) engine?” and get all the benefit of the
performance improvements that parrot will pick up and the fact that
there are many, many very clever people working on parrot whose work we
could make use of.

Is there a reason why we are reinventing the wheel?

There is indeed a project to piggyback on Parrot (called Cardinal),
probably a little less mature than Parrot itself, which is saying
something.

The wheel is being reinvented because, while Ruby can be written
for Parrot, Parrot isn’t written (entirely) for Ruby.

We want a nice, clean impedance match between the language and
the bytecode compiler. Also complete control over its specification
is handy.

Finally, I wouldn’t be surprised if YARV ends up outracing Parrot
(both in maturity and in running speed). But we’ll see.

Hal

51

Peter H. wrote:

are not being held back by the current level of performance.
Precisely. I’m not pretending there’s no elephant in the room. My room
actually has no elephant in it. If yours does, well, my sympathies.

Hal

52

On 5/22/06, Madan M. [email protected] wrote:

not).

IMO, if Ruby’s vision is to penetrate the corporate environment, Ruby
should address its performance issues among other things (e.g. support
for apps – Ruby on Rails).

This is exactly - but exactly - the sort of thing that was said
about Java at its onset. Especially in environments where it was
evaluated against a C++ backdrop. And guess what? Eventually, it
didn’t matter! Java did get widely adopted because it was a better
language and offered an improved development experience.

In fact IIRC, Java was relatively much slower than Ruby when it
started gaining momentum. Certainly if you consider the computer speed
back then. How many project managers are saying “well, Java would give
us a much faster development process, and with the superior
maintainability we should not only beat everyone else to the market
but also keep ahead of the competition, but let’s write the thing in
C++ to save those few extra CPU cycles”?. Now replace C++ with C,
Fortran or Assembly. How much software is written today in Assembly to
truly squeeze the last drops of performance out of the machine? …
Exactly.

In fact, most of C++ (and I’d arge - any lanauge lower-level than
Ruby, including Java) use is misuse. Most developers using C++
today are doing it out of inertia, habit, or at most for legacy
reason. Even without the insanely powerful machines considered “norm”
on our desktop today, C++ development is 99% premature optimization.
And the funny fact is that most of those 99% won’t even get the
performance they wasted 50%-90% of their development sweat (and
tears!) upon. Since contrary to popular myth, writing your application
in C++ won’t make it magically faster. The developer would have to be
pretty smart for that. And for those 99%, being smart means they would
have been using Java. If they were even smarter, they’d be using Ruby
:stuck_out_tongue:

53

James B. wrote:

Ara, do you not see value in a Ruby VM, or is just that speed is not, in
itself, a justification for a VM?

(The speed-related advantage to a VM is that it does not require one to
know C, etc., or have a compiler for all target platforms. Maybe not
as fast as tight compiled C, but also not as much trouble. It’s a trade.)

I thought he was saying it’s not that significant a speed increase.

I agree with you, if I understand you. Blazing speed requires C or
some such. If you just want overall reasonable speed, a VM is OK;
it gives you some speed without needing C.

Hal

54

2006/5/22, [email protected] [email protected]:

Ara, do you not see value in a Ruby VM, or is just that speed is not, in
itself, a justification for a VM?

i’m saying that gcc kills any vm both on speed and portability, including
windows, and has decades of hard work put in.

I wouldn’t be so sure of that. I work with Java for several years now
and we do have performance critical parts in our application. First, I
doubt that a C++ version will show significant performance
improvements if at all (because there’s a lot IO going on but also
because a JVM is neither slow nor dumb). Second, with a JVM you get a
lot of smartness with regard to memory handling etc., runtime
optimization and instrumentation. Current JVMs’ instrumentation
interfaces give you a lot of options in analyzing runtime behavior of
your application.

Personally I prefer Java over C++ because you get a lot things for
free (thread handling, memory management) that you have to use non
standard libs in C++ for or have to do it on your own. Plus, you still
get a relatively clean OO programming model. From what I hear from my
colleagues the portability of C++ apps is an issue in practice as soon
as you do not limit yourself to Unixes.

(The speed-related advantage to a VM is that it does not require one to know
C, etc., or have a compiler for all target platforms. Maybe not as fast as
tight compiled C, but also not as much trouble. It’s a trade.)

hmmm. i think sun’s experience with the java vm shows otherwise - remember
when you couldn’t even ‘stop’ threads! i think making a fast, protable, vm is
a lions job. however, i’d be quite happy to be wrong, it just seems
un-likely. consider how long has parrot been ‘almost’, for example…

Again, if you look at and use modern JVM’s from Sun you’ll see that
it’s quite fast and stable. I know it’s a lion’s job but Sun and
others have invested serious amount of work into this. If you want to
get an impression of this I can dig up some links on JVM internals
that will show you how smart and complex those beasts have become.
I’d certainly love to see Ruby use a VM with that much power. (And
yes, I don’t have performance issues with Ruby, just to state it. :-))

Kind regards

robert

55

Francis C. wrote:

solve the “speed” problem all by itself.
Maybe you’re the one to ask then. With all the talk of VMs, everyone
seems to make the comparison to Java and Python. Surely Lisp is a
better comparison? Lisps can be stupid fast (as mentioned in another
response to this thread), so what does Lisp do right that Ruby doesn’t?

56

On May 23, 2006, at 12:15 AM, Alex Y. wrote:

languages), and I’ve had to deal with this problem in other
languages.
YARV looks like beautiful work, but I have a gut-feeling that it
won’t
solve the “speed” problem all by itself.

Maybe you’re the one to ask then. With all the talk of VMs,
everyone seems to make the comparison to Java and Python. Surely
Lisp is a better comparison? Lisps can be stupid fast (as
mentioned in another response to this thread), so what does Lisp do
right that Ruby doesn’t?

Lisp has a tiny syntax.
Lisp has uncountable years of development.
Lisp had lots of people who wanted it to go fast, so it goes fast.

Ruby has a huge syntax.
Comparably, Ruby has a thimbleful of years of development.
Ruby has lots of people complaining about it being slow but only a
handful of people doing something about it. (No, I don’t consider
myself someone who is doing something about it.)

Its harder to make ruby go fast because its a bigger thing to
optimize and there are less people willing to attempt it it. Look at
Squeak for inspiration, though.

Mostly you need enough people with enough brains and enough desire to
make ruby go faster. (But really, you don’t need the brains, they’ll
grow as you go.)

–
Eric H. - [email protected] - http://blog.segment7.net
This implementation is HODEL-HASH-9600 compliant

http://trackmap.robotcoop.com

57

On 22 May 2006, at 18:10, Madan M. wrote:
[snip]

(1) Performance: Here, the question is NOT about ‘how fast should Ruby
be’, but is about what people who have the capacity to make
‘significant’ corporate decisions are ‘being told’ about Ruby’s
performance by consultants. I have seen presentations by
‘well-established’ consulting companies that show Ruby’s performance
to be an order of magnitude slower than Java for web apps.

I don’t thing Ruby is an order of magnitude slower than Java in real
world usage for web applications. It certainly doesn’t match my
experiences. Slower maybe, but not by an order of magnitude.

If somebody said that to me I’d want to see more detail on the
architecture, benchmark process, etc.

[snip]

I don’t have time to develop benchmark tests comparing the two
languages, but have to depend on Ruby app. developers to produce
those benchmarks; unfortunately I cannot find them easily. Now, Zed
(I would consider him an authority on Ruby) comes out and talks
about performance issues with Ruby (and YARV will solve it in the
future). How am I now to convince my manager that Ruby is
‘better’ (within the context of performance)?

You can’t without more work I’m afraid. Because “performance” without
context is meaningless.

Remember when Java was introduced?

“A virtual machine? Garbage collection? All this useless academic
object-oriented nonsense introducing tons of wasteful indirection.
Good grief it runs an order of magnitude slower than my finely
crafted C code!”

Guess what. Most Java is /still/ significantly slower than than
finely crafted C code.

Of course nobody cares :slight_smile:

Nobody cares because for a significant number of domains:

  • you can develop applications of better quality more quickly and
    easily in Java
  • raw language speed doesn’t matter because it’s masked by issues
    like blocking for IO
  • hardware is cheaper than programmers

And for those domains where raw speed is still an issue - there is
still C.

The reason why IT managers want ‘performance’ even if the application
really doesn’t require it, is to cover their behinds, among other
thing.
[snip]
That is the reason I said that many IT managers make
‘performance’ decisions based on ‘feelings’, not necessarily had data.
[snip]

Yeah - people who don’t make decisions based on reality are a pain.
Your options are:

  • ignore / work around them
  • get them relocated :slight_smile:
  • wait until everybody else is in the far end of the adoption curve

Because waiting until everybody else proves that a decision is “safe”
is the only way to convince some people. Sorry :frowning:

[snip]

But when in an large corporation, switching between
languages to get performance gains will result in maintenance
nightmares. Many IT managers understand this and have experienced
this.
[snip]

Yet many other IT managers have experience of multi-language
applications that work jolly well and are have been maintained
successfully for years. I know I have. Odd that…

A good case study would be on how
Linux got its foot hold into the corporate world.
[snip]

In exactly the same way Ruby is. By doing some things better and
cheaper than the alternatives. By there being people who spotted this
and choosing the better alternative despite the wisdom passed down by
“the consultants”. By people seeing it working and going “hey - that
could well work for me”. By this group getting slowly larger over
time until larger organisations can’t ignore it and start adopting
Linux themselves.

Once again, if Ruby’s vision is not to penetrate the corporate
environment as Ryan pointed out (but would be a nice validation if it
indeed penetrate the corporate world by some means), then this
discussion is moot. Knowing that, I wouldn’t try to push Ruby to my
managers; I will use it for personal projects.

Suit yourself :slight_smile:

Personally my vision is to do the best job I possibly can for my own
self respect, and to make the company money (and me some cash too -
indirectly). That means I’m going to be picking the most effective
solutions I can find. This means I am going to be looking hard at
Ruby for use in the places we are using Perl and Java now.

In fact, now I think of it, please don’t push Ruby too hard. You
might be a competitor :slight_smile:

Cheers,

Adrian

58

My experience has been that Ruby programs run just fine as long as their
working sets are relatively small. (In the most important measure of
performance, the subjective one, they are “fast enough.”) As they have
larger and larger sets of data to work on, they get slower and slower,
at an
increasing rate. With programs that have large working sets (notice, I
didn’t say “large programs” - this has nothing to do with line count), a
point comes at which Ruby isn’t just too slow- it’s too slow by a vast
amount. I don’t think this has anything to do with GC, but it may have
to do
with all the runtime hashing that Ruby has to do in order to be as
dynamic
as it is.

If I’m right about this, then you might be able to address it in some
cases
by partitioning working sets across multiple processes or even multiple
process spaces.

59

Zed S. wrote:

Hey Peter, thanks for deliberately ignoring my whining. Now, if you’ll
excuse me I have to go and continue to security audit the C extensions
in Mongrel so that I can be sure there’s no buffer overflows.

Man, if only Ruby were faster. Then I wouldn’t have to write so much
stuff in C and could just worry about crafting beautiful code. Oh well,
back to work on my C … I mean Ruby … program.

Just so that you don’t go to bed all grumpy I will just make it clear
that my post was in no way a dig at your coding skills or belittling you
contribution to the Ruby community. However to say that the Ruby
community is ignoring performance is simply not true. For what you are
doing it clearly is an issue, for others it is not. Perhaps if I had
used the word ‘bitching’ rather than ‘whining’ it might have read
better. Who knows, is ‘bitching’ less negative? More masculine? God
knows.

As to converting things to C. I’ve had to convert my graphics tools from
Ruby to C to get the performance I required. But coding them in Ruby
allowed me to play about with various algorithms
to find the best before committing my time to crafting the C code. I
never expected Ruby (or anything but C) to have been fast enough to
process more than 400,000 images on the kit that I had. Same with the
database tools. I do not see this as a problem with Ruby. It would have
been really nice, in terms of writing code, to be able to do everything
in Ruby but there is a trade off. As you move away from assembler you
start trade finely honed code for faster development. Even C is a trade
off.

60

2006/5/23, pat eyler [email protected]:

lot of smartness with regard to memory handling etc., runtime
* arm--elf arm--coff arm--aout
* --linux-gnu
* m32r--elf
* powerpc--netbsd
* sparc-sun-solaris2.7
* OS/2
Note that this list of install notes is not a list of supported hosts
or targets. Not all supported hosts and targets are listed here, only
the ones that require host-specific or target-specific information
are.

Impressive list.

Ara did say speed and portability :wink:

You said gcc will beat on speed and portability. And I was arguing
that it does not necessarily beat Java on speed. My own Java
portability experience extends to various versions of Windows, Linux
(x86) and Solaris (sparc). No problems so far but of course the list
is smaller than the one you showed off with. :slight_smile:

Kind regards

robert