Zed and Luis drop the bomb on Ruby's poor performance

I don’t have ParseTree installed on my (dooky) work machine, I think
I’ll try this at home tho.

But I have a question, apologies if it’s obvious – is that a
superficial result or a meaningful result? It looks as if the magic
missing link here is ParseTree. Can you just drop that output into a
Lisp interpreter, badda bing badda boom?

On May 24, 2006, at 5:08 PM, Christian N. wrote:

(scope
Now try to port the Ruby object system to CL so it will have
reasonable runtime performance.

Good luck.

I’m pretty sure CLOS is a super-set of the ruby object system. Only
thing I’m not sure of is Mixins and singleton methods, but I don’t
imagine Mixins to be too hard to add if necessary. One could just
define a module, and then when one included it in a class just have a
macro go down the list of methods in the Module and define them for
the class (basically #append_features). Singleton methods are harder,
but one could probably do what is done in ruby, (construct a class on
the fly that inherits from an objects real class and change the
objects class to this new class.). We already get handling all the
inheritance stuff for free (and arrays and strings, numbers etc.).

On May 25, 2006, at 10:04 AM, Giles B. wrote:

But I have a question, apologies if it’s obvious – is that a
superficial result or a meaningful result? It looks as if the magic
missing link here is ParseTree. Can you just drop that output into a
Lisp interpreter, badda bing badda boom?

Absolutely not. The sexp generated by ParseTree is ruby specific and
has nothing to do with lisp.


_why: zenspider’s most intense moments of solice are immediately
following the slaughter […]
_why: that topknot’s the only thing keeping a lid on the righteous anger
bricolage: yeah, that and his flagrant obsession with dvorak

On 22 May 2006, at 17:18, [email protected] wrote:

one! :slight_smile: ) for not focussing on Ruby’s performance - it’s
thanks,
and jolly expressive. I’ve no reason (at the moment) to require more
performance from it. When I switch back to doing graphics again,
I’ll be
generating my c, c++ or assembly with Ruby anyway, rather than hand
coding it, and it’ll probably be fast enough for that too.

Not to single you out (though I guess I am) but you are the first
one in the thread to write-in with “it’s good enough for me because
of .”

:slight_smile: Woah! Now hang on there.

I was giving a response to:
“Are there valid excuses for not focusing on improving Ruby’s poor
computational performance?”

And to paraphrase myself, the answer is:
Yes

However, as I said, I’ll be as happy as the next about any
performance improvements.

From what I remember CLOS has distinct class and method hierarchies,
and multi-dispatch of methods on all parameters. You can specialize
functions depending on almost anything, not just class of the first
parameter, which is the implicit receiver in message based OO systems.

I believe this is correct, and if I recall correctly it means that
CLOS is a superset of all normal OO systems. But I could be wrong.

ruby → lisp (for those who have not seen the news-group):

http://groups.google.com/group/comp.lang.ruby/browse_thread/thread/5d04fe293afcbcb9/771972335aae65cc?q=lisp&rnum=19#771972335aae65cc

But I have a question, apologies if it’s obvious – is that a
superficial result or a meaningful result? It looks as if the magic
missing link here is ParseTree. Can you just drop that output into a
Lisp interpreter, badda bing badda boom?

Absolutely not. The sexp generated by ParseTree is ruby specific and
has nothing to do with lisp.

so they didn’t add “puts” to Lisp while I wasn’t looking then. well,
that’s a bummer.

coming back to a brief moment of sanity, it’d probably be more useful
to write a Ruby VM in Lisp rather than a Lisp interpreter in Ruby
anyway. I mean if we’re still talking performance here.

2006/5/25, Giles B. [email protected]:

I don’t have ParseTree installed on my (dooky) work machine, I think
I’ll try this at home tho.

But I have a question, apologies if it’s obvious – is that a
superficial result or a meaningful result? It looks as if the magic
missing link here is ParseTree. Can you just drop that output into a
Lisp interpreter, badda bing badda boom?

You can drop it into Lisp and it’ll say, “that’s a nice list, but what
am I supposed to do with it?” What needs done it to write a set of
macros to expand the Ruby ParseTree output into Lisp code which does
much the same thing. It would be my first Lisp project, but with
people shouting about how easy Lisp syntax is to manipulate, and how
it is a superset of everything already, it shouldn’t be impossible to
do.

I guess the first thing to try and work out, is, which Lisp? I don’t
know them well enough to tell them apart, so my pick would be GNU
CLisp which I happen to have lying around anyway. Good choice?

Douglas

Unfortunately I’m a total Lisp n00b, but what I have on my machine @
home is a variant of CLISP. I don’t think it’s Gnu, but it is Common
Lisp, and as far as I can tell that is the most popular variant
around.

That thread in comp.lang.ruby is insane! In a good way.

On May 25, 2006, at 6:21 PM, Douglas L. wrote:

I guess the first thing to try and work out, is, which Lisp? I don’t
know them well enough to tell them apart, so my pick would be GNU
CLisp which I happen to have lying around anyway. Good choice?

Yes, it is a good choice I think. CLisp is very fast at ‘compiling’
new code so development is quick, but it does not have great
execution speed characteristics (so any benchmarks you do in CLisp
will be easily blown away by other implementations). Unless you get
into some dark corners of CL you can easily move to a faster lisp
later. The CLisp community is big and helpful (not to say the others
aren’t). Development environment might be a bit of a shock, you are
going to be moving right into the middle of the emacs world – so you
are looking at emacs and SLIME as your best option (if you are
already using vi/vim then you can get along but you’ll not have SLIME
(there’s a project underway to get SLIME and vim working together)).

Cheers,
Bob


Bob H. – blogs at <http://www.recursive.ca/
hutch/>
Recursive Design Inc. – http://www.recursive.ca/
Raconteur – http://www.raconteur.info/
xampl for Ruby – http://rubyforge.org/projects/xampl/

Logan C. [email protected] writes:

macro go down the list of methods in the Module and define them for
the class (basically #append_features). Singleton methods are harder,
but one could probably do what is done in ruby, (construct a class on
the fly that inherits from an objects real class and change the
objects class to this new class.). We already get handling all the
inheritance stuff for free (and arrays and strings, numbers etc.).

How do you handle method_missing?

On May 27, 2006, at 1:08 PM, Christian N. wrote:

imagine Mixins to be too hard to add if necessary. One could just

Douglas

Christian N. [email protected] http://
chneukirchen.org

Custom condition handler.

On May 22, 2006, at 9:50, James B. wrote:

Kirk H. wrote:

Or it’s the hamster in the living room. People don’t comment on it
because, for the most part, they don’t see it.

Those people having speed issues: Is it not a reasonable option to
replace the bottleneck Ruby code with custom compiled C libraries?

In a word, “no.” Not for me.

I’m a hobbyist programmer, not a professional. I program in order to
accomplish something else, or to save time, and I don’t spend more
than, oh, 4-5 hours a week, on average, programming. I cannot imagine
what I would ever program that would justify learning a whole new
language, when I haven’t even finished learning this one yet. (I know,
one never ‘finishes’ learning a language. But I still have to look up
“Array” and “Enumerate” methods, and probably always will, because I
haven’t memorized which one does what yet.)

On the other hand, it’s also amusingly irrelevant for me. I have
replaced some of my older code with newer Ruby versions in order to
make them faster. Ruby’s way faster than AppleScript, which is what
most of my older programs are written in.

On the third hand, I hope the next time this thread appears, nobody
will waste time advancing the nearly nonsensical argument that you can
always speed up ruby by replacing parts with custom compiled C
libraries. Or, to put it another way “You can make Ruby faster by . . .
not using Ruby.”

That might make my program run faster, but it didn’t do diddly-squat
for making Ruby faster. And a lot of this thread was occupied by
people who were distracted by this right answer to the wrong question.
Of course IT departments looking (rightly or wrongly) at the question
“How fast is Ruby?” will find this answer inadequate. Of course Zed is
underwhelmed by the joy of having to cross-code more of Mongrel from
Ruby to C. Of course non-native gems are generally less popular than
native ones. None of these ‘solutions’ make Ruby one iota faster.

If I use a non-Ruby library, I might be able to make my program go
faster, but I’m also incorporating into my program a chunk of code I
can’t rewrite, or even read. It also makes my program grossly less
portable. I can easily transport a 100% Ruby program to another system,
and even another platform, as long as they’ve got the right version of
Ruby installed.

Alas, the one program I currently need to move to other systems is
using a non-Ruby database interface. Fortunately, I only have to move
it to other binary-compatible systems, so I can embed the library in my
code. For a binary-incompatible system, I’m completely screwed, since I
cannot reasonably expect an end-user to install Ruby, Rubygems, this
particular gem, AND PostgreSQL on their desktop merely to use this cute
little program. The program requires the db driver, compiling the
driver requires pglib, and getting pglib involves installing/compiling
Postgres. (Or stealing it off some other binary-compatible system,
although maybe I just got lucky when I did that. {shrug})

So I encourage people to be mindful that speeding up a program that’s
written, in whole or in part, with Ruby is not the same as speeding up
Ruby itself. It’s great to have that option, and sometimes it’s the
optimal solution, but it’s not the same thing.

  1. YARV/Rite/whatever appears to be an impending scheme that will speed
    up Ruby itself. Continuing to refine and improve core Ruby code
    (whether that core code itself is written in Ruby or C or whatever) is
    another avenue to speed up Ruby.

  2. Effective, easy to use profilers/benchmarkers/analyzers, good/better
    documentation for same, and/or more/better signs pointing to such tools
    will help programmers make their own Ruby code go faster. Even if
    Ruby-the-language didn’t get any faster, if the average program, by the
    average programmer, is more likely to have been optimized (for speed or
    size or whatever that programmer cares about), that’s nearly the same
    thing: native Ruby code running faster than it used to.

  3. Touting or enhancing Ruby’s low-friction interfaces to other
    languages might make programs that incorporate some amount of Ruby run
    faster, but it also is effectively an admission of guilt. “Yes, Ruby
    really is as slow as you think it is; my advice is to use something
    else if speed matters to you.” That’s fine if/when Ruby really is too
    slow, but it looks to me (e.g. the counter-example of Rails) that
    there’s lots of ‘slow Ruby’ that could be faster if the code were
    slightly rewritten. This seems to be especially true when the specific
    issue is not “Ruby is too slow to do X” but rather “Ruby is slower than
    [put name of high-level language, not ‘C’, here]”

Based on this thread, and other messages I’ve read, Ruby’s speed
‘issue’ sure doesn’t sound like an elephant to me. I think it’s a cat.
For me, Ruby is soft, cuddly, and friendly, and more than rewards the
occasional scratches, hair balls, and litterbox scooping. But there are
some people with really nice furniture, allergies, and/or a small
apartment, and if more practical/useful profilers, or YARV, or whatever
can deodorize and automatically scoop the catbox, train Ruby not to
scratch the furniture, and cut down on the shedding, then I’m sure
they’ll be thrilled.

Dave H. wrote:

accomplish something else, or to save time, and I don’t spend more than,
oh, 4-5 hours a week, on average, programming. I cannot imagine what I
would ever program that would justify learning a whole new language,
when I haven’t even finished learning this one yet. (I know, one never
‘finishes’ learning a language. But I still have to look up “Array” and
“Enumerate” methods, and probably always will, because I haven’t
memorized which one does what yet.)

I would posit, then, that the discussion of speed here is probably not
the same for you as it is some others in the thread.

On the other hand, it’s also amusingly irrelevant for me. I have
replaced some of my older code with newer Ruby versions in order to make
them faster. Ruby’s way faster than AppleScript, which is what most of
my older programs are written in.

On the third hand, I hope the next time this thread appears, nobody will
waste time advancing the nearly nonsensical argument that you can always
speed up ruby by replacing parts with custom compiled C libraries. Or,
to put it another way “You can make Ruby faster by . . . not using Ruby.”

Interesting. You assert this to be a nearly nonsensical argument, yet
you describe yourself as hobbyist programmer, and say that, for you, the
speed concerns are amusingly irrelevant.

That might make my program run faster, but it didn’t do diddly-squat
for making Ruby faster.

Do you use the YAML parser in Ruby? Or the String class? They’re C.
Ruby and C are old friends. If you really want to know Ruby, you need
to know at least some amount of C. One of the great virtues of Ruby
is the way it hooks into C code. Suggesting that augmenting a Ruby
program with a C-based library is somehow un-Rubyish strikes me as,
well, nearly nonsensical.

Most of what I’ve seen in this thread are concerns that, at certain
times, for certain programs, certain people have certain speed issues.

I don’t think anyone said that Ruby itself was intrinsically slow for
every task, so for most of the people concerned with speed (people whose
concerns tend to arise from professional or academic circumstances),
suggesting the use of C sounds more than a little reasonable. It’s
pragmatic and professional.


James B.

“The use of anthropomorphic terminology when dealing with
computing systems is a symptom of professional immaturity.”

  • Edsger W. Dijkstra

On Jun 1, 2006, at 14:59, James B. wrote:

Interesting. You assert this to be a nearly nonsensical argument, yet
you describe yourself as hobbyist programmer, and say that, for you,
the speed concerns are amusingly irrelevant.

That’s correct. I’m sure it’s an error on my part to assume that you’re
implying that the fact I don’t find Ruby too slow for my own needs
somehow makes me incapable of logical thought.

That might make my program run faster, but it didn’t do
diddly-squat for making Ruby faster.

Do you use the YAML parser in Ruby?

No.

Or the String class?

Yes.

They’re C.

I find that assertion to be quite ludicrous. A C string is a
null-terminated byte sequence. I’ve happily never had to use to any
great extent any other language that didn’t effectively insulate me
from any need to track the terminal null (the vast majority don’t even
use that, but rather store the length as an integer at the front of
the string), or that allowed my code to inadvertantly overwrite the
terminal null, thus turning a string into an object of indefinite
length, sucking down whatever random bytes lay beyond the end of the
actual string bytes until it found some other “00x0” in memory.

The String class is not only nothing like a C string, it’s not a string
at all, in the sense that any/all other languages I’ve met use the
term. It’s an Object, and that makes it act in ways fundamentally
different than the strings I’ve met in SQL, AppleScript, Modula-2,
Pascal, or Basic (various dialects, all with line numbers), to name a
few.

Ruby and C are old friends. If you really want to know Ruby, you need
to know at least some amount of C.

You’re going to have to provide some rather compelling evidence before
I give that opinion much credence. Ruby and Perl are “old friends,”
too. Nevertheless, if there are aspects of Ruby that came from these
languages, and I cannot learn them directly and exclusively via Ruby,
that just says to me that the Ruby documentation still needs work. When
I decided that AppleScript (XCode and AppleScript Studio
notwithstanding) were inherently inadequate for the programming I
wanted to do, I selected Ruby in part because of how much NOT like Perl
it is.

Or perhaps I already know enough C. I spent part of a summer learning
C, and the most important thing I learned was how much I disliked the
decisions the language designers had made.

One of the great virtues of Ruby is the way it hooks into C code.
Suggesting that augmenting a Ruby program with a C-based library is
somehow un-Rubyish strikes me as, well, nearly nonsensical.

“Un-Rubyish?” I don’t believe I said any such thing. I was quite
specific. Claiming you can make Ruby faster by not using it, is
nonsensical. Claiming you can make a program written in Ruby by
replacing some part of it with another language is perfectly
reasonable. Confusing the two is sloppy reasoning.

For the record, although it’s irrelevant to my position, I agree that
Ruby’s ability to hook into C code is a strength of the language.

I don’t think anyone said that Ruby itself was intrinsically slow for
every task,

Let’s see. What was that quote that started this thread? Oh, yes. “I?ll
be honest right away though and say that Ruby is slow.”

Note the lack of qualifiers.

Happily, most people (including, as stated earlier, myself) find Ruby’s
speed, however it might compare to alternatives, more than adequate.

so for most of the people concerned with speed (people whose concerns
tend to arise from professional or academic circumstances), suggesting
the use of C sounds more than a little reasonable. It’s pragmatic and
professional.

I quite agree. Nor have I disagreed with this in any way. I will point
out, superfluously I hope, that the merits of extending Ruby’s
abilities, whether for speed or for some other reason, are quite
unrelated to my precis, as summarized in the subject line.

On Jun 1, 2006, at 6:56 PM, Dave H. wrote:

I find that assertion to be quite ludicrous. A C string is a null-
terminated byte sequence. I’ve happily never had to use to any
great extent any other language that didn’t effectively insulate me
from any need to track the terminal null (the vast majority don’t
even use that, but rather store the length as an integer at the
front of the string), or that allowed my code to inadvertantly
overwrite the terminal null, thus turning a string into an object
of indefinite length, sucking down whatever random bytes lay beyond
the end of the actual string bytes until it found some other “00x0”
in memory.

Um, he was saying how they are implemented in C.

You’re going to have to provide some rather compelling evidence
before I give that opinion much credence. Ruby and Perl are “old
friends,” too. Nevertheless, if there are aspects of Ruby that came
from these languages, and I cannot learn them directly and
exclusively via Ruby, that just says to me that the Ruby
documentation still needs work. When I decided that AppleScript
(XCode and AppleScript Studio notwithstanding) were inherently
inadequate for the programming I wanted to do, I selected Ruby in
part because of how much NOT like Perl it is.

He’s not saying ruby was influenced by C (like it was by perl) he is
saying C is the platform upon which ruby is built.

I think Dave’s point is that it’s irrelevant what Ruby itself is
implemented in. The problem is that whenever performance dicussions
come up, instead of discussing how to imrpove the speed of
Ruby-the-language, people instead talk about how easy it is to use
languages other than Ruby for programs that are supposed to be fast.

Dave wants to talk about how to make fast programs that are completely
written in Ruby. And I do too, though I have little to contribute
except to perhaps bring the discussion back in line. Maybe we could
talk about improving the speed of the Ruby implementation, or about
improving our use of the language. But can we have a discussion where
we pretend for a moment that we don’t all want to run out and
implement things in C?

-RYaN

On Fri, Jun 02, 2006 at 07:56:10 +0900, Dave H. wrote:

They’re C.

I find that assertion to be quite ludicrous. A C string is a
null-terminated byte sequence.

…snip…

The String class is not only nothing like a C string, it’s not a string at
all, in the sense that any/all other languages I’ve met use the term.

This is the point you should just let this one go I think. Ruby is
written
in C and the String object wraps one of these (as defined in ruby.h):

struct RString {
    struct RBasic basic;
    long len;
    char *ptr;
    union {
        long capa;
        VALUE shared;
    } aux;
};

What this boils down to is that a Ruby String is a C string with fancy
clothes on.

Cheers,
Phil

On 6/3/06, Ryan W. [email protected] wrote:

improving our use of the language. But can we have a discussion where
we pretend for a moment that we don’t all want to run out and
implement things in C?

-RYaN

It’s difficult to talk about the implementation of Ruby without
talking about C because the (most common) implementation is in C. The
work Koichi san is doing on YARV is great and I’m really looking
forward to Ruby 2.0 but the answer to many performance problems in
Ruby is still going to be to write C extensions. I’ll give the work
I’m doing on Ferret (port of Apache Lucene search library) as an
example. Search can be a performance critical portion of many
applications. Now the pure Ruby version of Ferret is about 100 times
slower than the version with the C extension. Now along comes YARV.
From what I’ve been hearing there is a 2-4 times speed up for some
applications. That still leaves an order of magnitude difference.

Now I’m not saying that writing C extensions is the answer to all your
Ruby performance woes. Most of the time, a better algorithm will do
the job. A faster implementation will help too. I will say however
that Ruby and C together make a powerful combination and I think C is
well worth learning. Not only will you be able to write C extensions
when you need to, but you’ll have a better understanding of the Ruby
internals which will in turn help you write faster Ruby code. And it’s
really not that difficult a language to learn.

C is not a dirty word,
Dave

Ryan W. wrote:

I think Dave’s point is that it’s irrelevant what Ruby itself is
implemented in. The problem is that whenever performance dicussions
come up, instead of discussing how to imrpove the speed of
Ruby-the-language, people instead talk about how easy it is to use
languages other than Ruby for programs that are supposed to be fast.

Dave wants to talk about how to make fast programs that are completely
written in Ruby. And I do too, though I have little to contribute
except to perhaps bring the discussion back in line. Maybe we could
talk about improving the speed of the Ruby implementation, or about
improving our use of the language. But can we have a discussion where
we pretend for a moment that we don’t all want to run out and
implement things in C?

-RYaN

I agree with Dave and Ryan.

This discussion should be about how to make “RUBY’s” performance faster
when you are using “RUBY”, not C.

I’m very well versed in the C/C++ language and I think C is a great way
to “extend” the language of Ruby, but C should not be a requirement to
get great performance from Ruby, this should already be available.

So let’s talk about what can be changed about Ruby to make it faster
out-of-the-box?

I don’t know the internals of Ruby (and I shouldn’t have to…yet), but
how about we start with the engine, rubyw.exe, are there some area’s
here that can be improved?

Thanks