Couple of Questions

[josetomas_r]

Hi everyone,

I was watching yesterday’s John L.'s Lang.NET presentation on
IronRuby (Great presentation, btw, really enjoyed it), and it was
pretty informative (to me, at least). Besides that, I’ve been looking
at the IronRuby and DLR code trying to become more familiar with how
everything fits together (haven’t tried my hand yet at extending the
libraries because I’m honestly a ruby newbie) and had a couple of
questions. I hope they’re not too stupid

The first one is related to RubySites. On the presentation, John shows
how the shared DynamicSites in RubySites are used, for example, to ask
if a given object/class supports a method and invoke it if it it does.
The example in the presentation was to_int() for converting an object
to an integer value, btw.
What I didn’t see John mention very explicitly is where/how those
shared DynamicSites get initialized so that they point to the right
places (maybe it’s dumb question and I’m just not getting it still!).

The second question was not directly influenced by the presentation,
and it’s basically if someone would care to explain a bit better how
Ruby obtains the corresponding RubyClass instance for an object. I
understand the basic mechanism at the code level, but some points I’m
not too sure about:

• What exactly is the criteria for deciding whether a given class is a
  singleton or not? I see this is brought up throughout the code, but
  isn’t very clear to me yet (and it might be a rubyism I’m not aware
  of).
• What’s the relationship between what goes around a RubyClass for a
  given type (in the case of .net objects) and the whole InstanceData
  business?

Thanks!

[josetomas_r]

Tomas R. wrote:

The first one is related to RubySites. On the presentation, John shows
how the shared DynamicSites in RubySites are used, for example, to ask
if a given object/class supports a method and invoke it if it it does.
The example in the presentation was to_int() for converting an object
to an integer value, btw.
What I didn’t see John mention very explicitly is where/how those
shared DynamicSites get initialized so that they point to the right
places (maybe it’s dumb question and I’m just not getting it still!).

Check out Martin Maly’s blog for an explanation of how DynamicSites
work.

http://blogs.msdn.com/mmaly/

This post in particular –
http://blogs.msdn.com/mmaly/archive/2008/01/19/building-a-dlr-language-dynamic-behaviors-2.aspx
– will probably answer your question.

[josetomas_r]

Hi Joe,

Check out Martin Maly’s blog for an explanation of how DynamicSites
work.

First of all, thanks for answering. I’m aware of Martin’s blog and
it’s a very nice resource, but honestly, the details still seem a bit
hazy to me. In particular:

Does this mean that a shared DynamicSite gets set up [1] for a given
class/object and specific method during the first Invoke() call? If
so, does calling DynamicSite.RespondTo() potentially setup the site
for a given target as well? (I’m guessing yes, otherwise I can’t quite
see how the code could possibly work, but it would be nice to have
confirmation).

Actually, this kind of brings another question:
As far as I can see, the shared DynamicSites are there to support the
IronRuby runtime/libraries themselves needing to use the ruby bindings
for specific methods, which is fine. But they are not involved when
the runtime is simply calling a method explicitly based on the code.

Take, for example, this simple piece of code:

x = Mine.new
print x.to_s
print x

Where Mine is some arbitrary class implementing a to_s() method.

Now, the first call is invoked using the normal InvokeMember action
semantics in IronRuby. The second one has an implicit to_s() call from
print, which actually uses the shared RubySites.ToSSharedSite site.
(it’s easy to check using the debugger that the first explicit call
doesn’t go through the shared site, as expected)

If so, then this would seem to imply that after this small snippet of
code executes there are at least two sites caching the rule for
Mine.to_s. Is my understanding of it correct? And if so, are there any
implications of that? (I can guess it might involve some memory
overhead).

As I said, I’m just trying to understand what’s going on in the code.
Feel free to whack me in the head with the rod of enlightenment if I’m
just asking stupid questions

[1] I realize the term here might not be the most appropriate.

[josetomas_r]

Shared DynamicSites don’t get initialized for each specific class/object
and method combination. A shared DynamicSite applies to all invocations
of a method like to_int. The DLR validates the context in which the
invocation occurred and then invokes the dynamic site’s target (a
delegate) for the object.

Rather than have tons of permutations of varying types baked into the
DLR, they chose to use rule sets (as described in Martin Maly’s blog) to
determine which operation(s) should be performed on objects to achieve
desired results and cache these rule applications so that it doesn’t
have to evaluate these rules over and over again (like, for instance, in
a loop where the same operation is being iterated many times).

Keep reading the code – RubyBinder.cs in the Ruby project and
DynamicSite.cs in the Microsoft.Scripting project – to see how this
happens for IronRuby. RubyBinder.cs should answer your question about
why the behavior is different between InvokeMember and ConvertTo.
Briefly, the Ruby action binder applied different rules to these
invocations.

Maybe one day someone with more acumen and expertise than myself will
write a book about how the DLR works internally like what Don Box, Jeff
Richter, and others did for the CLR. It’ll be easier to comprehend
then. In the meantime, you’ll just have to rough it on the bleeding
edge.

[josetomas_r]

If

so, does calling DynamicSite.RespondTo() potentially setup the site
for a given target as well?

Sigh; I’m an idiot; that happens for not looking again to see I was
really referring to RubySites.RespondTo() which uses yet another
shared dynamic site. The overall question still stands, though.

[josetomas_r]

Hi Joe,

Shared DynamicSites don’t get initialized for each specific class/object
and method combination.

I know; that’s why I mentioned “initialized” was probably the wrong
term to use. What I meant was that at some point in time, there would
be a rule created for a given class/method combination and somehow
associated with that DynamicSite.

Keep reading the code – RubyBinder.cs in the Ruby project and
DynamicSite.cs in the Microsoft.Scripting project – to see how this
happens for IronRuby. RubyBinder.cs should answer your question about
why the behavior is different between InvokeMember and ConvertTo.
Briefly, the Ruby action binder applied different rules to these
invocations.

That was pretty obvious. even to; but not really my point.

Maybe one day someone with more acumen and expertise than myself will
write a book about how the DLR works internally like what Don Box, Jeff
Richter, and others did for the CLR. It’ll be easier to comprehend
then. In the meantime, you’ll just have to rough it on the bleeding
edge.

I wasn’t particularly asking about the DLR internals; this isn’t
really the place to do that anyway (not that there’s actually one),
but rather about IronRuby’s use of it, and more specifically aimed at
understanding a bit better how the library implementations got hooked
into the runtime for their corresponding classes. I probably don’t
need to understand it anyway, but I prefer to.

[josetomas_r]

Tomas R.:

The first one is related to RubySites. On the presentation, John shows
how the shared DynamicSites in RubySites are used, for example, to ask
if a given object/class supports a method and invoke it if it it does.
The example in the presentation was to_int() for converting an object
to an integer value, btw.
What I didn’t see John mention very explicitly is where/how those
shared DynamicSites get initialized so that they point to the right
places (maybe it’s dumb question and I’m just not getting it still!).

Just looking over my slides again … I’m guessing that your question
relates to the RespondToSharedSite site that is called from RespondTo()?
If so, the initial version of that site contains absolutely nothing but
a call to UpdateSiteAndExecute(), which is the ‘cry for help’ method
that Martin talks about in his blog:
http://blogs.msdn.com/mmaly/archive/2008/01/22/building-a-dlr-language-dynamic-behaviors-3.aspx

So the first time through, the site will examine the method parameters,
perform the dynamic lookup, and cache the result of that dynamic lookup
in the site (in addition to invoking the target).

• What exactly is the criteria for deciding whether a given class is a
  singleton or not? I see this is brought up throughout the code, but
  isn’t very clear to me yet (and it might be a rubyism I’m not aware
  of).

Singleton classes in Ruby are classes that have exactly one instance.
They are constructed on the fly from an instance. eg

b,c = Bob.new, Bob.new

class << b

mess with a class only for b

def boo
end
end
b.boo
c.boo # fails

• What’s the relationship between what goes around a RubyClass for a
  given type (in the case of .net objects) and the whole InstanceData
  business?

Not sure about this question … can you clarify?

Thanks,
-John

[josetomas_r]

Hi John,

So the first time through, the site will examine the method parameters, perform the dynamic lookup, and cache the result of that dynamic lookup in the site (in addition to invoking the target).

Thanks, that was exactly the answer I was hoping for

Singleton classes in Ruby are classes that have exactly one instance. They are constructed on the fly from an instance. eg

Ahh thanks, that clarifies it!

• What’s the relationship between what goes around a RubyClass for a
  given type (in the case of .net objects) and the whole InstanceData
  business?

Not sure about this question … can you clarify?

I was referring to RubyExecutionContext.GetInstanceData() and
friends…

[josetomas_r]

Tomas R.:

• What’s the relationship between what goes around a RubyClass for
  a > given type (in the case of .net objects) and the whole
  InstanceData > business?

Not sure about this question … can you clarify?

I was referring to RubyExecutionContext.GetInstanceData() and
friends…

InstanceData is the per-instance data for Ruby object. For example,
instance variables (@abc), frozen/tainted flags, etc.

We try to be smart about storing it on the object for types that we
generate, but if it’s a .NET object sometimes we have to go through a
dictionary lookup to find the instance data for a given object.

• John

[josetomas_r]

Tomas R.:

We try to be smart about storing it on the object for types that we
generate, but if it’s a .NET object sometimes we have to go through a
dictionary lookup to find the instance data for a given object.

Is the object itself the key for that dictionary? (If yes, I’m guessing
this will affect the lifetime of said objects, right?)

Yup. It’s stored in a WeakReference though. See
InstanceDataDictionary.cs

• John

[josetomas_r]

Hi John,

InstanceData is the per-instance data for Ruby object. For example, instance variables (@abc), frozen/tainted flags, etc.

Thanks, good to know. That was my impression from looking at the code,
but wanted to make sure.

We try to be smart about storing it on the object for types that we generate, but if it’s a .NET object sometimes we have to go through a dictionary lookup to find the instance data for a given object.

Is the object itself the key for that dictionary? (If yes, I’m
guessing this will affect the lifetime of said objects, right?)

[josetomas_r]

Attached is a fully managed implementation of the skeleton classes and
methods from openssl.so that I posted last week.

I’m currently working on implementing digest.so, which is very similar.

Cheers, Wayne.

[josetomas_r]

First Problem:

If I define a new library class called say Digest::Class, ie a class
called “Class” within a module called “Digest” then when I run the
ClassInitGenerator it produces a method name clash between the
LoadClass_Class generated for this new class and the one generated for
the standard Ruby “Class”.

Second Problem:

I define a new extension class:

[RubyClass(“MD5”)]
public class MD5: Base
{
}

As part of the loading of this class I need to initialize a class
variable:
@@metadata = System.Security.Cryptography.MD5.Create();

Any idea how this initialization could/should be handled?

I’d be happy with a workaround for the moment.

In the longer term, rather than extending ClassInitGenerator to handle
this specific case, can I suggest that developers of such extension
classes be given the option to implement some kind of Init hook method
that will get executed as part of each classes’ definition process? This
would be equivalent to code that Ruby programmers can put inside a
class or module definition. This hook method would be passed the class
or module being initialized. We could then manually do whatever we
needed, eg attach singleton methods, define constants, set class
variables, etc.

Cheers, Wayne.

[josetomas_r]

I think it is reasonable to add custom initialization method for classes
to ClassInitGenerator. However, it is still better (for tooling,
reduction of duplicated code etc.) to initialize classes declaratively
(using attributes) than imperatively (executing arbitrary code).

The class name clash is a bug that we will fixed. ClassInitGenerator
doesn’t need to be used by all extensions but will be recommended.

As for testing, the preferred way to test libraries is to write RSpec
tests and contribute them to Rubinius. John will write more on specs.

Tomas

[josetomas_r]

Sorry, I realized after I posted that my issues are based on some
questionable assumptions …

Given that there isn’t yet a mechanism for loading ruby extension dlls,
I followed the workaround, used for example by the current socket class
implementations, where extension classes are simply added along side the
builtin classes defined in IronRuby.Libraries.

In the longer term, if non-builtin classes are implemented in separate
assemblies then my name clash problem becomes less of an issue. I was
also assuming that ClassInitGenerator would be used to automatically
generate the module initialization code, but this may not necessarily be
the case for non-builtin classes implemented in external assemblies. So
my per class initialization hook issue also goes away if developers are
simply able to manually implement their own equivalent of LoadModules().

It would be nice, however, to decide what this extention loading
mechanism is going to be.

Also, is there a simple way to test our C# extension libraries
standalone (ie without having to run rbx.exe)?
In the case of the openssl library it was easy to write a simple C# test
harness as the openssl implementation didn’t really use any of the
IronRuby infrastructure (apart from simple classes such as
MutableString). However, for more sophisticated libraries that, for
example make use of dynamic call sites, it is difficult to call the
library methods without the appropriate CodeContexts etc. Is there an
easy way to create the appropriate contexts etc for use in a simple test
harness or is it too difficult to bother attempting?

Cheers, Wayne.

[josetomas_r]

You may recall from an earlier post that one of the external libraries
used by some simple Rails use cases was Win32API.so

So, I started to look into how to implement this library - it was going
to be quite complex to implement in a fully managed way, involving lots
of dynamic pInvoke stuff.

I then thought, hey the same Rails code runs on other platforms that
obviously don’t support the Win32 API. So, upon further investigation I
learned that Rails tries to require “Win32API”, but if the attempted
load throws a not found exception it simply uses other more generic
mechanisms to achieve the task at hand.

So, implementing the Win32API for IronRuby is as simple as deleting
Win32API.so!

Cheers, Wayne.

[josetomas_r]

-----Original Message-----
From: [email protected]
[mailto:[email protected]] On Behalf Of
Tomas M.
Sent: Wednesday, 27 February 2008 8:03 AM
To: [email protected]
Subject: Re: [Ironruby-core] ClassInitGenerator issues

As for testing, the preferred way to test libraries is to
write RSpec tests and contribute them to Rubinius.

Yes, I can certainly write Ruby tests for these libraries as well,
but I’d prefer to also be able to write simple C# test harnesses that
directly
exercise these libraries.

Firstly, there’s the issue that external libraries can’t currently be
loaded into IronRuby.
And from a testing perspective, it’s always desirable to eliminate as
many variables as possible,
ie if a test doesn’t work, one doesn’t want to have to consider: is this
a bug in the library that I’m testing or in some other aspect of the
IronRuby infrastructure.

The OpenSSL_Test.cs file that I attached to a previous post is a simple
example of what I’m talking about. But I’m not sure how to extend that
approach when CodeContexts are required.

Cheers, Wayne.

[josetomas_r]

On Tue, 26 Feb 2008 21:04:08 -0700, Wayne K. [email protected]
wrote:

So, implementing the Win32API for IronRuby is as simple as deleting
Win32API.so!

Very cool!

Out of curiosity, given the fact that Rails has traditionally required
significant hair loss before getting it to run on Windows, why would
Rails
look for the Win32API library first before going about things another
way?

–
/M:D

M. David P.
Co-Founder & Chief Architect, 3rd&Urban, LLC
Email: [email protected] | [email protected]
Mobile: (206) 418-9027
http://3rdandUrban.com | http://amp.fm |
M. David Peterson

[josetomas_r]

Wayne K.:

So, implementing the Win32API for IronRuby is as simple as deleting
Win32API.so!

If only everything were so easy … thanks, Wayne!
-John