Ruby and E.V.E. Paradox

[casper_the_ghost]

Brian M. wrote:

Why not. Not a single word in that told you to go out of your way to
use it. Thanks goes wholly to OSS. Some of us like to innovate rather
than sit around. I don’t think I would be productive trying to modify
Subversion’s code base to do what I want out of an SCM tool. It just
comes down to the fact that I am better with Ruby.
Yes … I understand the need to create and need to innovate as much as
anyone. However, there are lots of worthwhile projects that are begging
for developers to enhance and maintain them while other developers are
re-inventing wheels, levers, inclined planes and pulleys. And there’s
even more “worthwhile” open source software that is languishing because
only other programmers can use it!

Now in the specific case of version control systems, the main reason
there seem to be so many of them is that different project management
schemes have different requirements, and only the heavy-duty industrial
strength (and expensive) ones like ClearCase have any hope of covering
lots of those bases. And before you go beating up on ClearCase, I don’t
happen to be a big fan of it, I know it has warts, but it’s what’s
mandated in a lot of places.

CVS is still “good enough” for lots of SCM users, and between CVS and
SVN, there’s not much need for more innovation as far as I’m concerned.
One of the big principles of “agile” and “pragmatic” development is
YAGNI – You Ain’t Gonna Need It!

sarcasm do

For that, why are we even speaking multiple of anything? Why don’t we
all use the same language, the same OS, the same hardware… and while
we are at it, we can have the same attitude ideas and personalities.

end
Well, actually, we pretty much do all use the same language – C and
dialects of C – and we pretty much all do use the same hardware – x86
and x86-64. We pretty much all have a “File” menu in the upper left
corner, we pretty much all have a mouse and a keyboard and a color
graphical display, we pretty much all talk to each other using IPV4 and
dozens of other standard protocols, etc., etc. Innovations need to be
compelling to get someone to use something else.

I don’t really think of Ruby as innovative – one of the charming things
about Ruby is that it is a collection of good ideas that have stood the
test of time. Regular expressions, classes, objects and methods, dynamic
“typing”, flexible syntax, “it’s always run time”, continuations,
arbitrary precision integer arithmetic, flexible arrays and hashes —
these are all wheels that Ruby didn’t re-invent!

–
M. Edward (Ed) Borasky, FBG, AB, PTA, PGS, MS, MNLP, NST, ACMC(P)
http://borasky-research.blogspot.com/

If God had meant for carrots to be eaten cooked, He would have given
rabbits fire.

[casper_the_ghost]

Hey Vincent,

Vincent F. wrote:

Am I alone in thinking this? Should I be looking at other embeddable
languages (Python, for example), or just go back and get the LUA code
working again? Am I doing something wrong?

Just an idea - have you ever looked at scheme ? It seems a good choice
for embedded scripts, although I never had to work with it so far.

Worth looking into.

I know no scheme at all, so it puts me at a disadvantage. However, a
quick search yields a few different interpreters, some which might
be suitable for embedding.

Thanks for that.

Garth

[casper_the_ghost]

gga,

gga wrote:

GD ha escrito:

I’m keeping things fairly simple; I mostly just bind
functions/methods
and wrap pointers in opaque data structures (eg. using
Data_Wrap_Struct
in the Ruby version). My plan was to store a pointer to the calling
class
in an opaque structure and store it in the corresponding Ruby class,
so
I can reference it when needed.

That’s possible, but it can get ugly. Do what SWIG does. Create a
hash that stores a mapping between the ruby object and the C++ object
(and/or viceversa). Every time you allocate a new class, you enter
it
in the hash. Every time the free function for the class is called,
the
mapping is removed from the hash.
You can even use ruby’s own hashes to do so.
That way, you don’t need to pass any pointer, and can access any
class
at any time from any function.
Also, if your classes contain or return pointers to other C++
classes,
you need to keep telling the GC about it. See the SWIG Zoo/Animal
tutorial in the Ruby section.

This is an interesting idea but may be overkill for what I’m trying
to do.

• The documentation is not at all clear how you protect something
  from the
  garbage collector, or indicate it is no longer in use.

Should some of the tricks used by SWIG be mentioned on the official
Ruby site, perhaps? Seems odd that an offsite source is considered
(mildly) authoritative.

Well, all that stuff IS properly documented in the PickAxe, in the
extending Ruby section. The PickAxe is somewhat poor in that it does
cover properly wrapping methods that return new C++ objects (the Zoo
and Animal tutorial in SWIG is a top-notch example of all the
headaches
you can run into on wrapping a complex library).

The online version of the PickAxe has been my main reference. It is
missing a bit of info, generally relating to marking and garbage
collection.

My secondary reference…

You also have this, if you haven’t read it.

ruby embedded into c++

I found it somewhat confusing, but it is probably another source.

… is this one.

This one fills in some of the gaps from the first. It is no literary
masterpiece but it is quite useful.

Okay, so just like an object. Good news. Very elegant and Ruby-ish.

Any way to grab the class from the Ruby script? What I really want is
to replace your first line with something like:

VALUE cFoo = rb_grab_class_from_script(“ClassDefinedInScript”)

Class names are just constants.
If your class is in the global namespace, use:

VALUE cFoo = rb_const_get( rb_cObject, rb_intern(“YourClass”) );

If not, replace rb_cObject for the proper module name.

This I didn’t know. So class names are basically constants and
can be accessed that way.

VALUE result = rb_protect( wrap_callback, VALUE(&data), &error );

You can do this!? Is this safe? It would seem to work on the idea
that VALUE is always going to be of sufficient size and structure
to hold a pointer directly. Is there any authoritative source that
says: “directly casting a pointer to a VALUE is perfectly fine and
will continue to be supported”.

If so, VALUE can be used (effectively) in the same way as a void*,
so there is no need to convert everything into Ruby objects as
I had thought.

If not, I’m not sure this call can be counted on to work reliably
in the future.

The Python docs look decent, I’d just rather not relearn Python
again,
I chose Ruby years ago, I’m good with it, and I like it better than
Python! But I’d rather move on from LUA if I can…

Python is worse at embedding than Ruby is. So if you have troubles
with Ruby, good luck with Python! On python you don’t need to just
take care of garbage collecting, but of reference counting. It is a
royal pain if you are doing it manually like you are.

I’ll probably give it a shot for interest’s sake. Looking closer at the
Python docs that don’t seem as strong as first impressions suggest,
although there are some decent examples.

I’ve spent close to two days on embedding Ruby and can’t get the
basic
things I’ve described working.

You should be able to get ruby up and running in about a couple of
hours (that’s all it took me).

Yes, I should (something basic anyway). Unfortunately, I didn’t (*).
This is the problem.

When such difficulties arise, they can suggest:

• it is the user’s fault; or
• there is a problem with the documentation; or
• the interface is unintuitive; or
• the library itself is buggy.

I shouldn’t have to explain why the likelihood of the first reduces
significantly with the number of external libraries the user has
successfully integrated in the past and the number of years they have
been working in software development. Any of the remaining three are
worth bringing to the attention of the dev gods.

Anyway I’m hoping that someone with the requisite knowledge spots these
posts, thinks “hey, perhaps we could look again at issue X” and
something comes of my experiences here. Or perhaps not, but nobody can
say I never mentioned it. Ultimately, the selfish aspect of my
motivation
is that someone spots these posts, and dispenses some form of guru-like
wisdom that instantly solves the problems I am having.

Googling some of the error messages I am seeing and issues I am facing
suggest I may not be alone.

(*) - Technically I did in a couple of hours, but it wasn’t stable and
crashed randomly, so I tried to figure out what was going wrong. No
luck.

Wrapping complex classes that return
pointers to other classes and the like, however, is hairy. But SWIG
is
there to help now.

I dearly want to be completely wrong.

You are.

That was unwarranted.

Anyway, in this exchange a number of issues have come up that an
experienced Ruby user (definitely not a guru though!) completely new
to the embedded interface tripped up on completely. I would suggest
that a good place for these little tips and tricks that keep coming up
would be somewhere near the official documentation, so that the next
person who comes along to play with embedded Ruby can start off
slightly further ahead than I did.

Garth

[casper_the_ghost]

Hey Ed,

VS is still “good enough” for lots of SCM users, and between CVS and
SVN,
there’s not much need for more innovation as far as I’m concerned. One
of the
big principles of “agile” and “pragmatic” development is YAGNI – You
Ain’t
Gonna Need It!

I think there’s heaps of room for new ideas and innovation in the SCM
area.
There are a fair number of tools floating around with various unique
features
and approaches. If the area was dead I think we’d just see one or two
tools gain
and hold dominance.

I don’t really think of Ruby as innovative – one of the charming
things
about Ruby is that it is a collection of good ideas that have stood the
test of
time. Regular expressions, classes, objects and methods, dynamic
“typing”,
flexible syntax, “it’s always run time”, continuations, arbitrary
precision
integer arithmetic, flexible arrays and hashes — these are all wheels
that
Ruby didn’t re-invent!

The appeal of Ruby to me is its incredible flexibility and the ability
to
develop small bodies of code in it really fast. There’s no fighting
the
compiler or dealing with odd legacy requirements. You aren’t pressured
into a
particular paradigm, and you can be sloppy if you want. It bundles a
whole bunch
of good ideas together and makes it available for you in a single
language. The
focus seems to be on flexibility and powerful expression, which leads to
elegant
solutions not always possible in other languages. You can experiment and
prototype incredibly quickly. It also scales well provided that you
don’t go too
far into the tens of thousands of lines of code.

I’d still go for a C+±like language for larger projects, because when
I’m
working on them I want the compiler to be absolutely pedantic and
complain about
every single thing it can, and I want complete control over how it
works. But
for many other things, Ruby is outstanding.

Garth

[casper_the_ghost]

Hey Ed,

M. Edward (Ed) Borasky wrote:

know the most about. Gambit has an experimental Erlang-like lightweight
process capability that might be good for game programming. I know next
to nothing about game programming. I don’t play the games and the last
time I even attempted to write one, the hardware was on the order of 64
kilobyte 6502s and the language of choice was Forth.

This is interesting. Sorry that I didn’t notice it when replying
to Vincent’s post. I’ll check these out.

And Gambit seems to be the name of a great many things. The Gambit
I know of is a companion to Fluent.

I’ve got a fair few points of reference for other embedded possibilities
now, quite ironic given the location. Anyway, off to research and
experiment.

Garth

[casper_the_ghost]

GD ha escrito:

This is an interesting idea but may be overkill for what I’m trying
to do.

Well, I’d suggest you read swig docs and see if you still think is
overkill for your project. Since you mentioned it is a game engine, I
seriously doubt that it will be overkill.

VALUE result = rb_protect( wrap_callback, VALUE(&data), &error );

You can do this!? Is this safe?

Yes. Again, read ruby.h. It checks for this to be true… or ruby
won’t compile.

Is there any authoritative source that
says: “directly casting a pointer to a VALUE is perfectly fine and
will continue to be supported”.

Besides me, you mean? I guess we have a trust issue, here

Well, that relies on a feature of the C/C++ language. As long as you
are using C ruby and we are dealing with 32/64-bit machines, it will
hold true. Again, read ruby.h and config.h.
Besides that, Matz has mentioned this is ok (search the list). His
goal for VALUE was to have it be like void*, but clearer to users and
keeping type safety. Unfortunately, C does not quite allow it to
behave like a void* in all contexts, so sometimes you do need to cast
it to void*.
This is more or less a limitation of C/C++.

If that’s not enough for you, I’ll try to see if I can convince the
Pope to say something along those lines, but it might be tricky

If so, VALUE can be used (effectively) in the same way as a void*,
so there is no need to convert everything into Ruby objects as
I had thought.

Err… you weren’t exposing each and every single little class in your
code, were you? Ouch.
That’s not the idea, of course. The stuff you need to convert to Ruby
is only the stuff you need your users to use in Ruby, as it stands to
reason.

I’ll probably give it a shot for interest’s sake. Looking closer at the
Python docs that don’t seem as strong as first impressions suggest,
although there are some decent examples.

Well… what EXACTLY are you trying to achieve, Garret? You started
the thread mentioning you WANTED to get away from Lua and use Ruby for
your game engine.
Personally, within the context of game engines, I must tell you, I
think that’s not such a good idea, as Lua is multi-thread safe today
which is definitively something you do want in a game engine (and
LuaJIT is probably THE fatest VM I have EVER seen for a non-static
typed language).
As such, I warned you before-hand that neither Python nor Ruby are
thread safe.
You did mention you liked Ruby’s syntax better than anything else and
you did not care about multi-threading, so I kept helping you with
ruby.
You mentioned you had difficulties embedding Ruby, and I gave you
pointers for you to read on. You did not understand how to use a
couple of functions, and I pointed to you the right stuff you should be
using.
You posted some code that crashed on you, and I gave you similar
working code that works reliably, with compiling instructions to boot.
You questioned whether my code was valid, which means you probably have
not tried running it.
You have also not followed my advice of looking at SWIG and, instead,
now you want to look at Python (!?). Are you pulling my chain?

If you DO want to use Python for wrapping a library, and you are
familiar with C++ template use, I suggest using boost::python and Py++.
It is slightly more efficient than SWIG, albeit it can be hard to
debug if you are not familiar with templates.
If you don’t like templates (like myself), I would ALSO recommend you
learn and use SWIG for embedding Python.
Is there any reason why you are avoiding SWIG and not even looking at
the documentation I pointed you to? In case you did not find it, it’s
here, btw:
SWIG and Ruby (read: memory management
section)
SWIG Users Manual

You will learn quite a lot about embedding stuff from reading it. My
guess is that the problems you are having with Ruby and perhaps Lua
might be because you are not too familiar with the subtleties that can
arise from exposing classes to scripting languages. SWIG does a really
good job of explaining that (best thing I’ve seen yet) and has working
and relatively well tested code in about 20+ languages.

The good thing about SWIG is that you can actually use Lua (which you
mentioned you know well already) and expose one or two classes that way
first. Then, you should be able to easily port that code to ruby, by
just changing just a couple of swig files (if any).
Also, if you end up dropping Ruby at some point in time, you should be
able to again port your code easily to whatever other language you
fancy.

(*) - Technically I did in a couple of hours, but it wasn’t stable and
crashed randomly, so I tried to figure out what was going wrong. No luck.

Well, if you’ve wrapped a class or two and they are crashing, your best
bet is posting the code to them to get help. Otherwise, it is unlikely
anyone will be able to help you out much.
Also, it will be hard for anyone to write better docs, if nobody knows
what exactly are you having difficulties with.

That was unwarranted.

True, it did read quite nasty once I read it myself. What I meant was
more like: “Luckily for you, you are wrong”.

I would suggest that a good place for these little tips and tricks that keep coming up
would be somewhere near the official documentation, so that the next
person who comes along to play with embedded Ruby can start off
slightly further ahead than I did.

Sure. Feel free to write them up somewhere. In case it was not clear,
any code or snippet of code I posted in this thread is public-domain.

[casper_the_ghost]

Hey gga,

gga wrote:

GD ha escrito:

This is an interesting idea but may be overkill for what I’m trying
to do.

Well, I’d suggest you read swig docs and see if you still think is
overkill for your project. Since you mentioned it is a game engine,
I
seriously doubt that it will be overkill.

I should clarify. My current usage of embedded scripting is fairly
basic. If I went the Ruby route, it would probably grow in scope in
time. Thus my original statement is only half correct. It is overkill
now, it might not be in the future.

I should also emphasise that the game engine handles the bulk of the
game stuff at present. The level scripts are mostly for tutorials or
small exceptions in level behaviour. Thus they are not terribly
demanding.

BUT…

As mentioned I’m weighing up expanding the scope of what the scripts
handle. Small baby steps first…

VALUE result = rb_protect( wrap_callback, VALUE(&data), &error
);

You can do this!? Is this safe?

Yes. Again, read ruby.h. It checks for this to be true… or ruby
won’t compile.

I remember a mention checking for the size of a number (long?) being
equivalent to a pointer, but not this specifically. Not saying it
isn’t there, I just don’t remember it.

Is there any authoritative source that
says: “directly casting a pointer to a VALUE is perfectly fine and
will continue to be supported”.

Besides me, you mean? I guess we have a trust issue, here

Not a trust issue. Just wondering if this is a “well, it works for
me and it’s good enough” or “this is the officially sanctioned way
to do it and it’d take a major release before it gets changed”.

Well, that relies on a feature of the C/C++ language. As long as you
are using C ruby and we are dealing with 32/64-bit machines, it will
hold true. Again, read ruby.h and config.h.
Besides that, Matz has mentioned this is ok (search the list). His
goal for VALUE was to have it be like void*, but clearer to users and
keeping type safety. Unfortunately, C does not quite allow it to
behave like a void* in all contexts, so sometimes you do need to cast
it to void*.
This is more or less a limitation of C/C++.

If that’s not enough for you, I’ll try to see if I can convince the
Pope to say something along those lines, but it might be tricky

I haven’t seen it in the list; but I haven’t searched it- because
I didn’t have any idea it could be the case at all until you
mentioned it. If it does have the Matz-seal-of-approval then that’s
good to know- is definitely worth mentioning somewhere in the
official docs as it makes a HUGE difference knowing you can just
cast any pointer to a VALUE rather than having to create an
artificial array to hold it.

And if an official thumbs-up or thumbs-down came along, that would
probably be of enormous benefit to others embedding Ruby.

If so, VALUE can be used (effectively) in the same way as a void*,
so there is no need to convert everything into Ruby objects as
I had thought.

Err… you weren’t exposing each and every single little class in
your
code, were you? Ouch.
That’s not the idea, of course. The stuff you need to convert to
Ruby
is only the stuff you need your users to use in Ruby, as it stands to
reason.

No way. No mass class exporting here. I’m actually content with
declaring
classes directly in Ruby and doing any needed conversion myself
(assuming
I can use opaque data structures too). I do want to expose a fair few
functions/methods from C++ to Ruby though, about twenty or so directly,
and many more indirectly. I need around a half dozen Ruby methods to
be visible to C++ for calling (the event and update methods I mentioned
previously).

I’ll probably give it a shot for interest’s sake. Looking closer at
the
Python docs that don’t seem as strong as first impressions suggest,
although there are some decent examples.

Well… what EXACTLY are you trying to achieve, Garret?

Garret is probably a lovely name to have. I also like Garnet as a
name. Unfortunately, around my birth I was named “Garth” and really
haven’t seen much reason to change it since.

You started
the thread mentioning you WANTED to get away from Lua and use Ruby
for
your game engine.

Yes, that would be great.

This is the progression of thoughts on the issue:

• I was a touch frustrated with LUA, so I thought I’d weigh up
  replacements.
• After spotting information on Embedded Ruby and skimming through the
  source,
  I thought it looked promising. Weighing things up, I decided I could
  get
  additional benefits from scripting with Ruby beyond the current
  limited
  scripting use. Gave myself two days.
• I gave it a shot. Didn’t turn out so well.
• Made a post detailing problems I’ve been having. Got a bunch of ideas.
  Plan to try many of them out to see which ideas work, and which
  don’t.

The thing of note is that I’m more willing to sink a good chunk of
time into getting Ruby going ahead of other languages, because I know
Ruby is a good language to work with. But the time isn’t open-ended. If
getting Ruby going would take 2 weeks and Python would take 2 hours,
my new scripting language is Python. If it was 2 days for Ruby and 2
minutes for Python, I’d go for Ruby. Basically Ruby gets extra grace
because I more fully understand what it can do.

Personally, within the context of game engines, I must tell you, I
think that’s not such a good idea, as Lua is multi-thread safe today
which is definitively something you do want in a game engine (and
LuaJIT is probably THE fatest VM I have EVER seen for a
non-static
typed language).

The interface to the script will always run in a single thread with
my software, so threading issues aren’t a huge concern. Power of
expression is very useful though, hence looking at Ruby.

As such, I warned you before-hand that neither Python nor Ruby are
thread safe.

Yep.

You did mention you liked Ruby’s syntax better than anything else and
you did not care about multi-threading, so I kept helping you with
ruby.

Yes.

You mentioned you had difficulties embedding Ruby, and I gave you
pointers for you to read on. You did not understand how to use a
couple of functions, and I pointed to you the right stuff you should
be
using.

Indeed, thanks.

You posted some code that crashed on you, and I gave you similar
working code that works reliably, with compiling instructions to
boot.
You questioned whether my code was valid, which means you probably
have
not tried running it.
You have also not followed my advice of looking at SWIG and, instead,
now you want to look at Python (!?). Are you pulling my chain?

I work two jobs. Both involve using Ruby. Unfortunately only one
involves Embedded Ruby. Factoring in any time difference, guess which
of the two jobs I’ve been at recently.

I’ll be back and able to do more Embedded Ruby stuff in under 24 hours.
I fully intend to pull in the sample code fragments you’ve been
mentioning
and see how they work with my code. As I mentioned, the SWIG suggestion
is good, and this is also something I will be playing with. Even if I
don’t use SWIG, no doubt I can learn much from its output.

I have not questioned your code, just indicated that I haven’t been
able to run it yet. My development and net access are on two
completely different non-connected machines, and my present Embedded
Ruby code is a mess- I’ve been experimenting to find ways around the
current problems I am experiencing. I can’t just drop your fixes in,
even if they are 100% perfect; my present Embedded Ruby code is still
a mess!

As for Python, this is what I used quite some time ago. Suddenly one
day a beautiful creature named Ruby came through and swept me off my
feet. Anyway, I don’t hate Python, I just like Ruby much more. So
it is still a valid choice for embedding target from my POV. Given
a choice, Ruby is the one for me. But I’m having real trouble with it.

So I’ll probably have a play around. I’ll see how I go with the new
Ruby knowledge I’ve picked up here, probably give Python a shot, try
out Scheme, and even give LUA another look. In each case there is
scope for improvement to my code. I do feel the greatest benefit will
come from Ruby though- just a matter of effort versus reward.

I’m basically going through an improvement/experimentation phase with
my software at the moment. Hence my desire to try a few things out.
I’m looking to spend some time now to get payoffs through the next
year.

If you DO want to use Python for wrapping a library, and you are
familiar with C++ template use, I suggest using boost::python and
Py++.
It is slightly more efficient than SWIG, albeit it can be hard to
debug if you are not familiar with templates.

If you don’t like templates (like myself), I would ALSO recommend you
learn and use SWIG for embedding Python.

Yep, SWIG seems like a good thing to examine regardless.

I didn’t know Boost had some Python gear. I’m using various bits of
Boost already. Could be interesting.

Is there any reason why you are avoiding SWIG and not even looking at
the documentation I pointed you to?

As above, lack of time for the next 24 hours. Then I get a few hours,
sleep, and then most of a day. Trust me, I’m not yet done with this
stuff! I’ve been given a bunch of useful ideas by various people on
this list (including yourself) and I feel I owe them the courtesy
of trying out some of the suggestions. It all takes time though.

In case you did not find it, it’s
here, btw:
SWIG and Ruby (read: memory management
section)
SWIG Users Manual

You will learn quite a lot about embedding stuff from reading it.
My
guess is that the problems you are having with Ruby and perhaps Lua
might be because you are not too familiar with the subtleties that
can
arise from exposing classes to scripting languages. SWIG does a
really
good job of explaining that (best thing I’ve seen yet) and has
working
and relatively well tested code in about 20+ languages.

The good thing about SWIG is that you can actually use Lua (which you
mentioned you know well already)

My LUA knowledge is “sufficient” only- I wouldn’t say I know it well.
I know Ruby well, and Embedded Ruby, barely at all.

and expose one or two classes that way
first. Then, you should be able to easily port that code to ruby, by
just changing just a couple of swig files (if any).
Also, if you end up dropping Ruby at some point in time, you should
be
able to again port your code easily to whatever other language you
fancy.

Bringing SWIG into my project seems an interesting idea. At this point
I’d rather learn from it, but perhaps it would be useful to use the
tool itself directly.

Re the mention of classes, I barely need to share classes at all. As
long as I can pass an opaque pointer to the game class through Ruby
and get it back at the other end, and call functions/methods from
C++ to Ruby to C++, I’m 98% of the way there.

The remaining 2% just involves some initialisation that involves
creating some Ruby objects on the C++ side and having them available
to manipulate with the functions and methods.

(*) - Technically I did in a couple of hours, but it wasn’t stable
and
crashed randomly, so I tried to figure out what was going wrong. No
luck.

Well, if you’ve wrapped a class or two and they are crashing, your
best
bet is posting the code to them to get help. Otherwise, it is
unlikely
anyone will be able to help you out much.

I’ve got a few snippets I can grab; the main things I’m still
uncertain of are relating to protecting objects etc from the
GC, and how to indicate they are free to clean. I’ll post them
as soon as I can.

Also, it will be hard for anyone to write better docs, if nobody
knows
what exactly are you having difficulties with.

Indeed. I’ve indicated (in length) some of the issues I’ve been having,
but you are right, I am light on the specifics.

I’d like to minimise the rest of the code as best I can to produce a
bearable test-case that causes problems because as it stands there is
too much non-Embedded-Ruby code for me to sensibly post (and I’m a bit
touchy about some of the non-Ruby code!).

My current code crashes (in the second rb_gc) with something of the
form:

void a()
{
init_ruby();
rb_gc();
}

void b()
{
rb_gc();
}

int main(int argc, char **argv)
{
a();
b();
}

And starts working when I reorder like this:

void a()
{
rb_gc();
}

void b()
{
rb_gc();
}

int main(int argc, char **argv)
{
init_ruby();
a();
b();
}

This triggered my question about whether Ruby makes assumptions about
the
stack and if it needs to be called in a persistent frame (not this is
not
the case in the first sequence). The actual code of course is much more
complex; we’ve got calls that cross library boundaries and a few more
stack layers in-between. As mentioned, I’d like to reduce the problem
down
to something simpler. But given the simplified code above, could you see
a potential crash from running it like that? If so, that is something
potentially worth mentioning.

Reducing the code down further to a more minimalistic form may expose a
problem on my side as well. If not, I’ll have something sensible to
provide as a “is something wrong with this code” example.

I would suggest that a good place for these little tips and tricks
that keep
coming up
would be somewhere near the official documentation, so that the next
person who comes along to play with embedded Ruby can start off
slightly further ahead than I did.

Sure. Feel free to write them up somewhere. In case it was not
clear,
any code or snippet of code I posted in this thread is public-domain.

The best I could manage is to collate your thoughts, as is quite
apparent
my Embedded Ruby knowledge is extremely limited. I am completely and
utterly unqualified to write about Embedded Ruby! There are topics
I could write on, but when restricted to Ruby I could at best offer
tutorials to new users. The existing ones are already pretty good.

Now it’s 11:30pm, so I’m off to bed. Hopefully I’ll be back to embedded
language experimentation next evening after my other job.

Garth

[casper_the_ghost]

Hey gga,

gga wrote:

My current code crashes (in the second rb_gc) with something of the form:

Post your ACTUAL code that crashes.

Of course. Reasons why I haven’t done this yet in my previous
post. Will post when I can get a reasonably minimal set with
problems.

init_ruby() does not exist as an
api call in ruby, so we’ll need to see source to that.

My mistake, init_ruby should be ruby_init, it was late and
I was going from memory.

rb_gc();
}

int main(int argc, char **argv)
{
a();
b();
return 0;
}

I can tell you it works fine for me,

Okay, that’s good news. I have a working side and a failing side
to work from now, let’s find where they meet in the middle.

Garth

[casper_the_ghost]

An additional note: I’ve since been running the code provided for
several hours
in mode 3 (Ruby and messy stack frame), built without profiling
information
(ie. without “-pg”) and it has been running without failure for over
2600
iterations. If I rebuild with “-pg” the problem generally strikes in the
100-200
iterations range. Thus it is probably a fair assumption that the problem
arises
only when profiling is enabled. I’ll leave it running for a bit longer
and will
post any failures should they occur.

Garth

[casper_the_ghost]

My current code crashes (in the second rb_gc) with something of the form:

Post your ACTUAL code that crashes. init_ruby() does not exist as an
api call in ruby, so we’ll need to see source to that.
If what you meant was something like this:

/**

• @file embed.cpp
• @author gga
• @date Wed Jan 17 11:40:27 2007
• g++ -I/usr/lib/ruby/1.8/x86_64-linux embed.cpp -o embed -lruby1.8

*/

#include <ruby.h>

void a()
{
ruby_init();
rb_gc();
}

void b()
{
rb_gc();
}

int main(int argc, char **argv)
{
a();
b();
return 0;
}

I can tell you it works fine for me, so if that crashes for you may
have a broken ruby or some .so dependency issue on your machine.

[casper_the_ghost]

• Does the test code look okay? Did I make any mistakes? I’m only human, and
  the crash test isn’t trivial.

Nothing caught my eye at first glance, but I have not yet looked at it
deeply.

• Does “./foo 3” also crash on your machine? Or is it just mine?

I cannot get it to crash on my box (64-bits, thou)… 750+ and counting

• I use alloca to simulate calls to functions with different stack depth
  requirements. If you object to alloca, please note that mode 2 of the
  test program works perfectly fine- this is heavy alloca use without Ruby
  calls. It crashes only when we add Ruby calls to the mix.

Use of alloca can be problematic if you are using latest SVN ruby
(according to a recent ChangeLog). Ruby provides some ALLOCA macros
for the same behavior, called ALLOCA_N() and in the new ruby’s
C_ALLOCA, I believe. Also, these macros will invoke the gc in case of
lack of memory, which is also a good idea, anyway.

• As noted, I think it only happens if profiling is enabled. I can’t guarantee
  it doesn’t crash without, I just haven’t seen it do so.

A simpler possibility for this is that ruby might be running out of
stack space in your process.
The current ruby interpreter is notoriously bad for the way it
allocates its stack frames. If you end doing a lot of deep nested
calls, you can hit the limits.
As you are on linux, you can test this rather easily.

Try increasing the value of ulimit, from its default.

ulimit -s
ulimit -s XXXX # like double your previous limit

See if the crash happens at the same place, or if it occurs somewhat
later (or not at all).
If that’s what happens, I’m afraid you’ll need to modify the process
stack depth for your application, or always run your code thru a bash
wrapper script that sets ulimit.

[casper_the_ghost]

Hey everyone,

Tracking down this problem was NOT easy, but here we go.

I created a small program that made some basic embedded API calls,
mirroring
the functionality I was trying to add into E.V.E. Paradox. It worked
perfectly, suggesting the problem was not Ruby.

I then took the existing E.V.E. Paradox code and tried to add traces and
extra
calls in to find the problem. Basically I added calls that repeatedly
called
rb_funcall at various points. Basically at various points in my code it
was
possible to call rb_funcall successfully, branch off into other code,
then
make the exact same call again and have it crash.

Tracking it down further was a nightmare. Basically the point of the bug
moved
as extra calls were added to perform traces and call rb_funcall. The
probability of a crash tended towards zero in any region of code as more
traces and calls were added. Thus as I got closer to the problem, it
would
move somewhere else. It suggested some kind of stack dependency from
Ruby or
corruption being caused in my code or the libraries I was using.

For a while I thought the cause was a poor interaction between Ruby and
freetype, as it most commonly crashed after calls relating to font
manipulation when inside that library.

However, I was finally able to create some isolated code that crashed
with
high frequency. So it’s not E.V.E. Paradox or freetype. I did this by
writing
a function that simulates API calls being made in the context of a
larger
program through controlled recursive calls and alloca. Basically the
call
wraps various Ruby interactions inside and alongside repeated adding and
removing of stack frames through nested calls.

This code is included below (NB: Linux only).

Interestingly enough, I haven’t seen it crash yet when profiling is
disabled.
Only when it is enabled. Is there some negative interaction between the
profiling code in the gcc 4.* series and Ruby? Anyway, I thought it
interesting.

As for running the code, simply build it with make (you may need to fix
the
path to Ruby) and run it in one of these four modes:

./foo 0

This will run a simple set of code that outputs an series of annoying
traces. No stack trickery is done. It works.

./foo 1

This will run a simple set of code that calls an embedded Ruby script,
which calls a callback in the code. It works.

./foo 2

This performs the same traces but uses “messy” to fire them off within
differing stack contexts. It is slow, but works.

./foo 3

This performs the same set of Ruby calls, but uses “messy” to fire them
off
within differing stack contexts. It is slow, and eventually crashes.

Thus Embedded Ruby works fine in simple code, but put it in the context
of
a larger body and it has the potential to crash.

At some point I will have a play around with different versions of Ruby
to
see if they also trigger this crash, and if I can work around the
problem
this way.

Anyway, what I’d like to ask of the ordinary Ruby users like me and of
the
Ruby Gods is:

• Does the test code look okay? Did I make any mistakes? I’m only human,
  and
  the crash test isn’t trivial.
• Does “./foo 3” also crash on your machine? Or is it just mine?
• Are there any Ruby Gods for whom the code crashes with the skill to
  diagnose the problem with Ruby itself? I fear this may be beyond me
  (but
  I’ll be giving it a shot).

Other info:

• The rest of E.V.E. Paradox and the multiple external libraries work
  fine
  with profiling enabled. IF it is a compiler/profiler bug, it
  affects Ruby
  only.
• My development machine is pretty stable, quite able to perform
  days-long
  tests and compiles consistently- although I have to admit the DVD
  tray
  mechanism is a bit touchy. This isn’t a memory or solar flare thing.
• I use alloca to simulate calls to functions with different stack depth
  requirements. If you object to alloca, please note that mode 2 of the
  test program works perfectly fine- this is heavy alloca use without
  Ruby
  calls. It crashes only when we add Ruby calls to the mix.
• As noted, I think it only happens if profiling is enabled. I can’t
  guarantee
  it doesn’t crash without, I just haven’t seen it do so.
• “Don’t enable profiling” isn’t really an acceptable workaround when it
  works fine for the rest of the large body of code and external
  libraries
  and my development build includes it by default. I really don’t want
  to
  have to rebuild the whole project when I’m after timing
  information…
  Besides, “does not work with profiling” appears to be a significant
  bug.

Anyway, this is enormously frustrating, as after all of this struggle
I’ve
been able to put together a proof-of-concept that I can use the
embedded
Ruby interface in my code, but I cannot actually keep it due to the
stability issues. :(

I am available for questions, further information, and testing should
anyone
with the requisite knowledge to diagnose the problem need my input.
Please
feel free to drop me an email if I can help (see the contact page at
www.entropicsoftware.com for my direct email).

Further detailed information below. Apologies in advance for the huge
post,
but there is a lot of relevant detail to convey.

Garth

---

============================================================================
== Contents of foo.cpp:

#include <ruby.h>

#include
#include
#include
#include

#define MYRAND(r) (rand()%r)

using namespace std;

static VALUE obj;

typedef void (MessyFunc)(void *);

typedef VALUE (*RubyGenericFunc)(…);

static int mode = 0;
static bool use_messy;
static bool use_ruby;

static VALUE RubyP(int argc, VALUE *argv, VALUE self)
{
VALUE str = rb_str_new(“”, 0);
for (int i=0; i<argc; i++)
{
if (i > 0)
rb_str_cat(str, ", ", 2);
rb_str_concat(str, rb_inspect(argv[i]));
}
cerr << "Ruby P: " << RSTRING(str)->ptr << “\n”;
return Qnil;
}

static VALUE RubySafeGCX(VALUE a)
{
rb_gc();
return Qnil;
}

static VALUE RubySafeGC()
{
int status;
VALUE rv = rb_protect(RubySafeGCX, Qnil, &status);
if (status)
{
VALUE einfo = rb_obj_as_string(ruby_errinfo);
cerr << "Ruby reported error: " << status
<< " " << RSTRING(einfo)->ptr << “.\n”;
}
return rv;
}

static void RubySafeGCZ(void *t)
{
RubySafeGC();
}

static void InitRuby(void *t)
{
ruby_init();
RubySafeGC();
}

static void InitRuby2(void *t)
{
rb_define_global_function(“p”, (RubyGenericFunc)RubyP, -1);
}

static void InitRuby3(void *t)
{
int status;
rb_eval_string_protect(
“class Foo\n”
" def initialize(z)\n"
" @a = z\n"
" end\n"
" def update(v)\n"
" p(v)\n"
" end\n"
“end\n”
, &status);
if (status)
{
VALUE einfo = rb_obj_as_string(ruby_errinfo);
cerr << "Ruby reported error: " << status
<< " " << RSTRING(einfo)->ptr << “.\n”;
}
}

static void InitRuby4(void *t)
{
VALUE c_foo = rb_const_get(rb_cObject, rb_intern(“Foo”));
VALUE c_bar = INT2NUM(555);
obj = rb_funcall(c_foo, rb_intern(“new”), 1, c_bar);
}

static void update(int i)
{
rb_funcall(obj, rb_intern(“update”), 1, INT2NUM(i));
}

static void update2(void t)
{
update(((int *)t));
}

static void badger(int i)
{
cerr << (((i % 6) < 4) ? "Badger " : “Mushroom”) << " " << i <<
“\n”;
}

static void badger2(void t)
{
badger(((int *)t));
}

static void Nothing(void *t)
{
}

static void messy2(MessyFunc t, void *td, int depth, bool &done)
{
// Add random crap to the stack that will be checked and reclaimed at
// the end.
int ss = MYRAND(40)*4+8;
int mv = MYRAND(256);
unsigned char *data = (unsigned char *)alloca(ss);
memset(data, mv, ss);

for (int i=0; i<ss; i++)
{
if (data[i] != mv)
{
cerr << “Stack corrupted immediately.\n”;
cerr << (int)(data[i]) << " " << mv << “\n”;
exit(1);
}
}

//cerr << "D: " << depth << “\n”;
if (depth < 0)
return;

for (int i=0; i<(MYRAND(3)+1); i++)
messy2(t, td, depth-1, done);
if ((depth == 0) && (!done) && (t))
{
(*t)(td);
done = true;
}
for (int i=0; i<(MYRAND(3)+1); i++)
messy2(t, td, depth-1, done);

for (int i=0; i<ss; i++)
if (data[i] != mv)
{
cerr << “Stack was corrupted.\n”;
cerr << (int)(data[i]) << " " << mv << “\n”;
exit(1);
}
}

void messy(MessyFunc t, void *td = NULL)
{
int depth = MYRAND(5)+6;
bool done = false;

if (use_messy)
messy2(t, td, depth, done);
else
{
if (t)
(*t)(td);
}
}

int main(int argc, char *argv[])
{
if (argc != 2)
{
cerr << “Usage: foo \n”;
cerr << " 0 - Simple Badger.\n";
cerr << " 1 - Simple Ruby.\n";
cerr << " 2 - Badger with stack use.\n";
cerr << " 3 - Ruby with stack use.\n";
exit(1);
}
mode = atoi(argv[1]);
use_messy = (mode & 2);
use_ruby = (mode & 1);

cerr << "Mode " << mode
<< ". Messy " << (use_messy ? “enabled” : “disabled”)
<< ". Ruby " << (use_ruby ? “enabled” : “disabled”)
<< “.\n”;

cerr << “Empty test…\n”;
messy(Nothing);

// Initialise.
cerr << “Initialising…\n”;
if (use_ruby)
{
messy(InitRuby);
messy(RubySafeGCZ);
messy(InitRuby2);
messy(RubySafeGCZ);
messy(InitRuby3);
messy(RubySafeGCZ);
messy(InitRuby4);
messy(RubySafeGCZ);
}

// Test over and over.
cerr << “Thrashing…\n”;
for (int i=0; i<10000; i++)
{
if (use_ruby)
{
messy(RubySafeGCZ);
messy(update2, &i);
messy(RubySafeGCZ);
}
else
messy(badger2, &i);
}

// Clean up.
cerr << “Done thrashing.\n”;
if (use_ruby)
{
messy(RubySafeGCZ);
}

cerr << “Everything was okay.\n”;
}

============================================================================
== Contents of Makefile:

foo: foo.o
g++ -Wall -pg -g -o foo foo.o -L/packages/ruby/lib -lruby

foo.o: foo.cpp
g++ -Wall -pg -g -c -o foo.o foo.cpp
-I/packages/ruby/lib/ruby/1.8/i686-linux

clean:
rm -f foo.o foo *~

============================================================================
== Extra Notes

uname -a
Linux notimportant 2.6.18-1.2798.fc6 #1 SMP Mon Oct 16 14:54:20 EDT 2006
i686
athlon i386 GNU/Linux

cat /etc/fedora-release
Fedora Core release 6 (Zod)

ldd foo | grep ruby
libruby.so.1.8 => /packages/ruby/lib/libruby.so.1.8
(0x00110000)

ls -la /packages/ruby/lib/libruby.so*
lrwxrwxrwx 1 root root 16 Jan 15 23:19
/packages/ruby/lib/libruby.so →
libruby.so.1.8.5
lrwxrwxrwx 1 root root 16 Jan 15 23:19
/packages/ruby/lib/libruby.so.1.8 →
libruby.so.1.8.5
-rwxr-xr-x 1 root root 1897472 Jan 15 23:18
/packages/ruby/lib/libruby.so.1.8.5

ls -la /usr/lib/libruby.so*
ls: No match.

echo $LD_LIBRARY_PATH | grep /packages/ruby/lib | wc -l
1

g++ -v
Using built-in specs.
Target: i386-redhat-linux
Configured with: …/configure --prefix=/usr --mandir=/usr/share/man
–infodir=/usr/share/info --enable-shared --enable-threads=posix
–enable-checking=release --with-system-zlib --enable-__cxa_atexit
–disable-libunwind-exceptions --enable-libgcj-multifile
–enable-languages=c,c++,objc,obj-c++,java,fortran,ada
–enable-java-awt=gtk
–disable-dssi --enable-plugin
–with-java-home=/usr/lib/jvm/java-1.4.2-gcj-1.4.2.0/jre
–with-cpu=generic
–host=i386-redhat-linux
Thread model: posix
gcc version 4.1.1 20061011 (Red Hat 4.1.1-30)

ruby -v
ruby 1.8.5 (2006-12-25 patchlevel 12) [i686-linux]

##########################

Ruby was built from source with:

tar xfz …/ruby-1.8.5-p12.tar.gz
cd ruby-1.8.5-p12 || exit 1
./configure --enable-shared --prefix=/packages/ruby-1.8.5-p12 || exit 1
make || exit 1
make install || exit 1

Also note:

ls -lad /packages/ruby
lrwxrwxrwx 1 root root 14 Jan 15 23:19 /packages/ruby → ruby-1.8.5-p12

##########################

make
g++ -Wall -pg -g -c -o foo.o foo.cpp
-I/packages/ruby/lib/ruby/1.8/i686-linux
g++ -Wall -pg -g -o foo foo.o -L/packages/ruby/lib -lruby

./foo 3

Mode 3. Messy enabled. Ruby enabled.
Empty test…
Initialising…
Thrashing…
Ruby P: 0
Ruby P: 1
[ … lines removed … ]
Ruby P: 124
Ruby P: 125
(eval):7: [BUG] Segmentation fault
ruby 1.8.5 (2006-12-25) [i686-linux]

Abort

##########################

gdb ./foo
GNU gdb Red Hat Linux (6.5-8.fc6rh)
Copyright (C) 2006 Free Software Foundation, Inc.
GDB is free software, covered by the GNU General Public License, and you
are
welcome to change it and/or distribute copies of it under certain
conditions.
Type “show copying” to see the conditions.
There is absolutely no warranty for GDB. Type “show warranty” for
details.
This GDB was configured as “i386-redhat-linux-gnu”…Using host
libthread_db
library “/lib/libthread_db.so.1”.

(gdb) set args 3
(gdb) run
Starting program: /home/garthy/dev/test/ruby/foo 3
Mode 3. Messy enabled. Ruby enabled.
Empty test…
Initialising…
Thrashing…
Ruby P: 0
Ruby P: 1
[ … lines removed … ]
Ruby P: 124
Ruby P: 125

Program received signal SIGSEGV, Segmentation fault.
0x004935fe in st_lookup (table=0x0, key=6905, value=0xbfd83c08) at
st.c:250
250 hash_val = do_hash(key, table);
Current language: auto; currently c
(gdb) bt
#0 0x004935fe in st_lookup (table=0x0, key=6905, value=0xbfd83c08) at
st.c:250
#1 0x0042aa7c in search_method (klass=3085892280, id=6905,
origin=0xbfd83c38)
at eval.c:480
#2 0x0042aadd in rb_get_method_body (klassp=0xbfd83c68, idp=0xbfd83c74,
noexp=0xbfd83c78) at eval.c:501
#3 0x00434263 in rb_call (klass=3085892280, recv=3085892320, mid=6905,
argc=1, argv=0xbfd83ca0, scope=1) at eval.c:6024
#4 0x00434866 in vafuncall (recv=3085892320, mid=6905, n=1,
ar=0xbfd83d04)
at eval.c:6125
#5 0x004349d0 in rb_funcall (recv=3085892320, mid=6905, n=1) at
eval.c:6142
#6 0x08048ce3 in update (i=126) at foo.cpp:101
#7 0x08048d02 in update2 (t=0xbfd8440c) at foo.cpp:106
#8 0x080491bd in messy2 (t=0x8048cea , td=0xbfd8440c, depth=0,
done=@0xbfd843cf) at foo.cpp:150
#9 0x08049151 in messy2 (t=0x8048cea , td=0xbfd8440c, depth=1,
done=@0xbfd843cf) at foo.cpp:147
#10 0x08049151 in messy2 (t=0x8048cea , td=0xbfd8440c, depth=2,
done=@0xbfd843cf) at foo.cpp:147
#11 0x08049151 in messy2 (t=0x8048cea , td=0xbfd8440c, depth=3,
done=@0xbfd843cf) at foo.cpp:147
#12 0x08049151 in messy2 (t=0x8048cea , td=0xbfd8440c, depth=4,
done=@0xbfd843cf) at foo.cpp:147
#13 0x08049151 in messy2 (t=0x8048cea , td=0xbfd8440c, depth=5,
done=@0xbfd843cf) at foo.cpp:147
#14 0x08049151 in messy2 (t=0x8048cea , td=0xbfd8440c, depth=6,
done=@0xbfd843cf) at foo.cpp:147
#15 0x08049151 in messy2 (t=0x8048cea , td=0xbfd8440c, depth=7,
done=@0xbfd843cf) at foo.cpp:147
#16 0x0804934b in messy (t=0x8048cea , td=0xbfd8440c) at
foo.cpp:171
#17 0x08049611 in main (argc=2, argv=0xbfd844b4) at foo.cpp:223

[casper_the_ghost]

Hey gga,

gga wrote:

• Does “./foo 3” also crash on your machine? Or is it just mine?

I cannot get it to crash on my box (64-bits, thou)… 750+ and
counting

Interesting. Mine is pretty-much guaranteed to die between 100 and 200
iterations. Generally if it makes it that far, it’s going to make it the
whole way.

For sake of comparison, what do you get on that same box for:

ruby -v
g++ -v
uname -a

• I use alloca to simulate calls to functions with different stack
  depth

requirements. If you object to alloca, please note that mode 2 of
the

test program works perfectly fine- this is heavy alloca use
without Ruby

calls. It crashes only when we add Ruby calls to the mix.

Use of alloca can be problematic if you are using latest SVN ruby
(according to a recent ChangeLog). Ruby provides some ALLOCA macros
for the same behavior, called ALLOCA_N() and in the new ruby’s
C_ALLOCA, I believe. Also, these macros will invoke the gc in case
of
lack of memory, which is also a good idea, anyway.

I hope the use of an alternate alloca is merely an internal thing and
doesn’t impose an external requirement on code using embedded Ruby.
Requiring that the host program not use alloca would be pretty harsh.
I generally don’t use it (though I did for my example code), but I can’t
guarantee the other libraries I am also using don’t use it. As such,
that
might be the limitation in common. I hope not…

• As noted, I think it only happens if profiling is enabled. I can’t
  guarantee

it doesn’t crash without, I just haven’t seen it do so.

A simpler possibility for this is that ruby might be running out of
stack space in your process.
The current ruby interpreter is notoriously bad for the way it
allocates its stack frames. If you end doing a lot of deep nested
calls, you can hit the limits.
As you are on linux, you can test this rather easily.

Try increasing the value of ulimit, from its default.

ulimit -s
ulimit -s XXXX # like double your previous limit

This was a really good idea, I was crossing my fingers and hoping it
would work.

Alas, it still fails at around the same spot with the limit set to
10240, 81920, 163840, and 655360. No major behavioral change.
Original was 10240.

Since I normally launch via tcsh, I issued a “limit” and got 10240
(same as bash). To be sure, I issued a “limit stacksize unlimited”, and
gave it one last run. It doesn’t help, still dies at 125.

It sounded feasible though. Stack-based errors, profiler information may
flood stack usage, and put in a large program- it could quite easily
have
been the cause.

Thanks for giving things a whirl.

Current plan: Trying out a few different versions of Ruby to see the
effect.
Hopefully I have some more information tomorrow.

Garth

[casper_the_ghost]

Hey all,

Okay, I have some more information.

Re the use of alloca in foo.cpp, I remembered a GNU extension that I
rarely
use, and it can be safely swapped in. In foo.cpp, replace:

unsigned char *data = (unsigned char *)alloca(ss);

with:

unsigned char data[ss];

I generally prefer to keep my code portable, but this serves to shift
the
potential blame away from alloca somewhat. The sample code still
crashes,
just not in the same point. alloca is not to blame.

I’ve rebuilt the same foo.cpp code with multiple versions of Ruby and
tabulated the results:

(best viewed in fixed-width)

Version Mode 3 Mode 1

---

1.8.5-p12 125
1.8.4 91
1.6.8 0 0
Stable repos snapshot 125
Nightly checkout snapshot 0 0

Snapshots were downloaded on 22/1/2007, SA Central time (SA).

In each case any minor modifications required to make it build (eg.
fixing
prototypes) were made.

In all cases the sample code (foo.cpp) crashed.

Mode 3 refers to the iteration that it crashed when run with “messy”.
Mode 1 refers to crashes when run on its own, ie. without “messy”.
A result of 0 means it crashed before it started iterating. Blank means
it
was not run- I only bothered in the cases where Mode 3 crashed quickly.

The gist is that the sample code provided will crash on every version I
tried (1.6.8, 1.8.4, 1.8.5-p12, latest stable and unstable CVS). It is
fair
to assume it will fail on many more.

Approaching the problem from another angle, I figured I’d try different
tricks when building Ruby itself. For the following tests I used
1.8.5-p12
as a base.

Modification Mode 3 Mode 1

---

Base (for comparison) 125
Link with static version instead ~90 [1]
CPPFLAGS=“-g -pg” 19
Strip out -O2 0 153 (!?!?)
Replace -O2 with -O 18

[1] - Forgot to record actual number, but it was around 90 or so.

And last of all, I just wanted to confirm that when running in the
default
configuration without “-pg”, the sample code made 10000 iterations
without
a crash. Took most of the day to run too.

Other pertinent information from my previous post:

uname -a
Linux notimportant 2.6.18-1.2798.fc6 #1 SMP Mon Oct 16 14:54:20 EDT 2006
i686
athlon i386 GNU/Linux

g++ -v
Using built-in specs.
Target: i386-redhat-linux
Configured with: …/configure --prefix=/usr --mandir=/usr/share/man
–infodir=/usr/share/info --enable-shared --enable-threads=posix
–enable-checking=release --with-system-zlib --enable-__cxa_atexit
–disable-libunwind-exceptions --enable-libgcj-multifile
–enable-languages=c,c++,objc,obj-c++,java,fortran,ada
–enable-java-awt=gtk
–disable-dssi --enable-plugin
–with-java-home=/usr/lib/jvm/java-1.4.2-gcj-1.4.2.0/jre
–with-cpu=generic
–host=i386-redhat-linux
Thread model: posix
gcc version 4.1.1 20061011 (Red Hat 4.1.1-30)

Maybe it is a gcc 4.* thing? Anyone else running a similar system (ie.
FC6)
able to give things a shot?

So there we go. I’m not sure what else I can provide at this point. I’ve
provided sample code that triggers the crash, tried it on multiple
versions
of Ruby, isolated it down to the -pg switch, so forth. I believe I’ve
fielded
any concerns on the sample code adequately. This is a fairly significant
bug
for anyone embedding Ruby in a project that needs profiling information-
assuming it affects other people as well- as it seems to affect multiple
versions of Ruby and makes the program unreliable and crash-prone.

My offer to assist remains open; I can be reached via email from the
contact
page on www.entropicsoftware.com. I do hope that the days of work I’ve
put
into diagnosing this problem somehow pays off in locating the source of
this
significant bug.

Now I feel it is time to take a break from Embedded Ruby and play around
with
some alternate solutions. Working on this has certainly been an
interesting
experience.

Please don’t hesitate to ask if I can provide any additional information
or
data from additional tests.

Take care,
Garth