Possible bug in org.jruby.anno.JavaMethodDescriptor and handling of static methods

dubstep · February 28, 2012, 12:34am

version is 1.6.7

The following class gives me an error when I try to load it, and I posit
that it shouldn’t:

package org.apache.pig.scripting.jruby;

import org.jruby.Ruby;
import org.jruby.RubyClass;
import org.jruby.RubyModule;
import org.jruby.RubyObject;
import org.jruby.anno.JRubyClass;
import org.jruby.anno.JRubyMethod;
import org.jruby.runtime.ObjectAllocator;
import org.jruby.runtime.ThreadContext;
import org.jruby.runtime.builtin.IRubyObject;

@JRubyClass(name=“ExampleA”)
public class RubyExampleA extends RubyObject {
private static final ObjectAllocator ALLOCATOR = new
ObjectAllocator() {
public IRubyObject allocate(Ruby runtime, RubyClass klass) {
return new RubyExampleA(runtime, klass);
}
};

public static RubyClass define(Ruby runtime) {
    RubyClass result =

runtime.defineClass(“ExampleA”,runtime.getObject(), ALLOCATOR);

    result.kindOf = new RubyModule.KindOf() {
        public boolean isKindOf(IRubyObject obj, RubyModule type) {
            return obj instanceof RubyExampleA;
        }
    };

    result.defineAnnotatedMethods(RubyExampleA.class);

    return result;
}

public RubyExampleA(final Ruby ruby, RubyClass rc) {
    super(ruby,rc);
}

@JRubyMethod
public IRubyObject initialize(ThreadContext context) {
    return this;
}

@JRubyMethod
public static IRubyObject hey(ThreadContext context) {
    System.out.println("hey");
    return context.nil;
}

}

The error:

NativeException: java.lang.NegativeArraySizeException: null
from org/jruby/anno/JavaMethodDescriptor.java:77:in <init>' from org/jruby/RubyModule.java:591:inclump’
from org/jruby/anno/TypePopulator.java:62:in populate' from org/jruby/RubyModule.java:685:indefineAnnotatedMethodsIndividually’
from org/jruby/RubyModule.java:573:in defineAnnotatedMethods' from org/apache/pig/scripting/jruby/RubyExampleA.java:30:indefine’
from org/apache/pig/scripting/jruby/PigJrubyLibrary.java:18:in
load' from ./pigudf.rb:5:in(root)’

Digging around a little but, in org/jruby/anno/JavaMethodDescriptor.java
I
see:

    int start = (hasContext ? 1 : 0) + (isStatic ? 1 : 0);
    int end = parameters.length - (hasBlock ? 1 : 0);
    argumentTypes = new Class[end - start];

So in the case where it is static and has no arguments, you’re going to
have issues. I tried removing the static piece and adding an argument,
and
predictably, it was fine.

Would love to hear if this is expected, and what I should do to mitigate
it
if it is… and if it’s not, where I should file a bug report

makki · February 28, 2012, 2:21am

On Mon, Feb 27, 2012 at 5:34 PM, Jonathan C. [email protected]
wrote:

@JRubyMethod
public static IRubyObject hey(ThreadContext context) {
    System.out.println("hey");
    return context.nil;
}
}
…

Would love to hear if this is expected, and what I should do to mitigate it
if it is… and if it’s not, where I should file a bug report

There should be an earlier, better error message, and I would love for
you to submit a patch to add such a thing. But this is expected.

In the case of a static method being bound to a Ruby method, it does
still have an argument: self. The instance method case has that as an
implicit argument since it’s called on the object itself, but static
methods don’t have that. In the case of a static Java method bound as
a Ruby class method, the argument will be the class itself. For a
static Java method bound as a Ruby instance method, the argument will
be the object instance.

In essence, “self” is the only non-optional argument for a
@JRubyMethod-bound method; it’s just handled for you by the JVM in the
case of an instance method.

The error should indicate that static methods still need to receive a
“self” of some kind.

I’m willing to debate whether it should be permitted to have a static
method that doesn’t even receive “self”, but it doesn’t work that way
at present.

Charlie

makki · February 28, 2012, 8:48am

Ahh, ok, this makes sense.

2012/2/27 Charles Oliver N. [email protected]

makki · February 28, 2012, 6:43pm

Charles,

Perhaps I am misunderstanding something, so I’d appreciate your help on
this again. I implemented the class at the bottom[1]

The issue is that I am not able to refer to this statically. Perhaps I
am
misunderstanding the use of static in JRuby classes? My goal is to
replicate the following on the JRuby side:

class A
def self.hey
puts “hey”
end
end

Now, I’m not sure if a static method in Java is what replicates that or
not… I appreciate your help on the matter!

This is more stuff that I shall try and put into the wiki!

[1]
package org.apache.pig.scripting.jruby;

import org.jruby.Ruby;
import org.jruby.RubyClass;
import org.jruby.RubyModule;
import org.jruby.RubyObject;
import org.jruby.anno.JRubyClass;
import org.jruby.anno.JRubyMethod;
import org.jruby.runtime.ObjectAllocator;
import org.jruby.runtime.ThreadContext;
import org.jruby.runtime.builtin.IRubyObject;

@JRubyClass(name=“ExampleA”)
public class RubyExampleA extends RubyObject {
private static final ObjectAllocator ALLOCATOR = new
ObjectAllocator() {
public IRubyObject allocate(Ruby runtime, RubyClass klass) {
return new RubyExampleA(runtime, klass);
}
};

public static RubyClass define(Ruby runtime) {
    RubyClass result =

runtime.defineClass(“ExampleA”,runtime.getObject(), ALLOCATOR);

    result.kindOf = new RubyModule.KindOf() {
        public boolean isKindOf(IRubyObject obj, RubyModule type) {
            return obj instanceof RubyExampleA;
        }
    };

    result.defineAnnotatedMethods(RubyExampleA.class);

    return result;
}

public RubyExampleA(final Ruby ruby, RubyClass rc) {
    super(ruby,rc);
}

@JRubyMethod
public IRubyObject initialize(ThreadContext context) {
    return this;
}

@JRubyMethod
public static IRubyObject hey(ThreadContext context, IRubyObject

arg) {
System.out.println(“hey”);
return context.nil;
}
}

2012/2/27 Jonathan C. [email protected]

makki · February 28, 2012, 11:04pm

On Tue, Feb 28, 2012 at 12:39 PM, Jonathan C. [email protected]
wrote:

def self.hey

import org.jruby.runtime.builtin.IRubyObject;
RubyClass result =
return result;

@JRubyMethod
public static IRubyObject hey(ThreadContext context, IRubyObject arg) {

System.out.println(“hey”);
return context.nil;
}
}

Have you ever tried meta annotation for static method?

@JRubyMethod (meta = true)
public static IRubyObject some_class_method(ThreadContext context) {
…
}

-Yoko

makki · February 28, 2012, 11:40pm

Yoko,

I have not. I would deeply appreciate it if you could perhaps explain
what
the meta flag signifies. Also, will that affect how it is inherited?

Thanks!
Jon

2012/2/28 Yoko H. [email protected]

makki · February 29, 2012, 3:23am

I’m glad I’m not the only one for whom “Whether this method should be
defined on the metaclass.”
is pretty hard to grok Perhaps this flags that it needs to receive
itself as an argument?

In fact, I guess I might as well ask the next question, though I can’t
make
code examples because I’m still sorting out the above confusion… given
that in Ruby, class methods are inherited, but in Java they are not, how
do
you deal with that? IE you have the following ruby:

class A
def self.hey
puts “hey”
end
end

class B<A
end

How would you implement B in java such that it works, without the super
lame static reimplementation that calls super.hey?

Thanks for the help, Yoko

2012/2/28 Yoko H. [email protected]

makki · February 29, 2012, 2:54am

On Tue, Feb 28, 2012 at 5:30 PM, Jonathan C. [email protected]
wrote:

Yoko,

I have not. I would deeply appreciate it if you could perhaps explain what
the meta flag signifies. Also, will that affect how it is inherited?

This is “Whether this method should be defined on the metaclass.”
according to API doc of org.jruby.anno.JRubyMethod,
http://jruby.org/apidocs/ . I myself want a clearer explanation about
this setting.

From my experience of developing pure Java Nokogiri, which is a JRuby
extension, meta is necessary to define class methods. Ruby’s class
methods are static methods in Java.

From the nature of a class/static method, those are not inherited by
descendants.

-Yoko

makki · February 29, 2012, 6:28pm

Charles,

This is beginning to make much more sense. So then the next question is,
given the code below[1], could you implement this or replicate on the
java
side? Given the snippet [2], since the methods are both static, there’d
be
no way for B to inherit the method “x” on the Java side, so the only way
to
replicate the code in [1] would be to explicitly inherit it? Is there
any
way to replicate not just the action but the intent of the ruby code, or
is
this just a case of “this is why writing code in Ruby and not in Java
can
often be nice”?

Thanks again!

[1]
class A
def self.x
self.y
end

def self.y
puts “yo”
end
end

class B < A
def self.y
puts “nah”
end
end

[2]
@JRubyClass
public class A {
@JRubyMethod(meta=true)
public static IRubyObject x(IRubyObject self) {
y();
}

@JRubyMethod(meta=true)
public static IRubyObject x(IRubyObject self) {
System.out.println(“hey”);
}
}

2012/2/28 Charles Oliver N. [email protected]

makki · February 29, 2012, 7:07pm

On Wed, Feb 29, 2012 at 11:27 AM, Jonathan C. [email protected]
wrote:

Charles,

This is beginning to make much more sense. So then the next question is,
given the code below[1], could you implement this or replicate on the java
side? Given the snippet [2], since the methods are both static, there’d be
no way for B to inherit the method “x” on the Java side, so the only way to
replicate the code in [1] would be to explicitly inherit it? Is there any
way to replicate not just the action but the intent of the ruby code, or is
this just a case of “this is why writing code in Ruby and not in Java can
often be nice”?
…
class B < A
def self.y
puts “nah”
end
end

Assuming your intent is to have A’s x() call B’s y() when called
against B, you’re right…there’s no way to do this without also
implementing a new x() in B. In Java, static methods don’t really live
on any object (they’re basically global functions that happen to be
namespaced within a class), so there’s no equivalent to polymorphism
like you want here. In Ruby, class methods exist on a metaclass
hierarchy where polymorphism works the same as it does on any class
hierarchy.

Charlie

makki · February 29, 2012, 6:04am

On Tue, Feb 28, 2012 at 8:22 PM, Jonathan C. [email protected]
wrote:

I’m glad I’m not the only one for whom “Whether this method should be
defined on the metaclass.”
is pretty hard to grok Perhaps this flags that it needs to receive itself
as an argument?

I can explain it this way…

Whether static or not, all methods on a class passed to
defineAnnotatedMethods are defined on that class…unless you
explicitly specify that they should go on the metaclass with meta =
true. The meta flag is provided as a convenience so you can have Java
instance methods and static methods get bound as Ruby instance and
class methods.

meta = true is necessary because it’s also possible to define Ruby
instance methods with Java static methods, like this:

public class MyObject extends RubyObject {
@JRubyMethod
public static IRubyObject foo(IRubyObject self) {
MyObject myObject = (MyObject)self;
…

This allows locating methods for a given class anywhere, and
additionally allows the use of defineAnnotatedMethods to define
methods for a type that does not have a Java class associated with
it…

RubyClass someCls = runtime.defineClass(“SomeClass”);
someCls.defineAnnotatedMethods(SomeClassMethods.class);
…
public static class SomeClassMethods {
@JRubyMethod
public static IRubyObject foo(IRubyObject self) {
…

Does that help clarify things?

end

class B<A
end

How would you implement B in java such that it works, without the super lame
static reimplementation that calls super.hey?

Just define it on A’s metaclass, either with the meta = true flag or
by doing the following:

RubyClass aClass = runtime.defineClass(“A”, runtime.getObject(), …);
aClass.getSingletonClass().defineAnnotatedMethods(AClass.class);
RubyClass bClass = runtime.defineClass(“B”, aClass, …);
…
public class AClass {
@JRubyMethod
public static IRubyObject key(IRubyObject aClass) {
…

JRuby handles the metaclass hierarchy, so it should show up on B in Ruby
too.

If you really want to be able to call it from Java against B, you have
to do the super-lame thing, since as you point out Java does not
inherit static methods.

Charlie

makki · February 29, 2012, 7:51pm

On Wed, Feb 29, 2012 at 1:05 PM, Charles Oliver N.
[email protected] wrote:

often be nice"?

namespaced within a class), so there’s no equivalent to polymorphism
like you want here. In Ruby, class methods exist on a metaclass
hierarchy where polymorphism works the same as it does on any class
hierarchy.

My understanding about Ruby’s class method was wrong. But, I’m bit
confused what you want to do.

I think we should code in Ruby side as much as possible. When we come
to the point unable to code in Ruby or Java does far better than Ruby,
we should implement only that part by Java as an extension. This is
like C extension for CRuby. At least, the first method to get started
should be written in Ruby. For example, method x should be written in
Ruby, and method y is in Java. Making y a private method is a good
idea.

The problem happens when B inherits A and overrides y method.
Technically, Ruby can do that, but JRuby with Java extension can’t do.
My question is who will override y. If you (Jonathan) have designed
such API, probably, you can avoid that. If a user might do, you can
make y method private. Then, unexpected behavior by overriding private
method will be user’s responsibility as well as not a recommended way.

-Yoko

makki · March 1, 2012, 5:58am

On Wed, Feb 29, 2012 at 7:27 PM, Jonathan C. [email protected]
wrote:

That makes perfect sense. I guess it boils down to “the niceness of ruby is
why it is worth investing millions of lines of code of Java to put it on the
JVM”

Well I wouldn’t say millions, but yeah

Totals grouped by language (dominant language first):
java: 194136 (97.37%)
yacc: 3783 (1.90%)
ruby: 1455 (0.73%)

Ruby code

what I’d love to do is somehow compile pig.rb and put it into pig.jar such
that I can just do require ‘pig.jar’ in a ruby script, and still get the
same library. Do you know if that is possible?

If pig.rb is at the root of pig.jar, doing this will get you the same
files:

require ‘pig.jar’
require ‘pig.rb’

Obviously having separate names might be better, but any jar that’s
loaded into JRuby also gets added to the load path, so it’s pretty
simple to package things together.

Charlie

makki · March 16, 2012, 1:39am

Charles,

This was very helpful. Is there any way to make it so that “require
‘pig.jar’” would implicitly “require ‘pig.rb’”?

2012/2/29 Charles Oliver N. [email protected]

makki · March 16, 2012, 5:56am

Well maybe 1.7 is a good time to talk about this.

In the past, you could name a class in a special way and we’d load it
when you require the jar. In your example:

require ‘pig.jar’

…would look for a class name PigService in the default (toplevel)
package. If found, it would instantiate and load that service.

I hate this pattern for a couple reasons:

toplevel classes are blowful
if you have the jar in a nested path, like require ‘foo/pig.jar’,
the class it looks for is foo.PigService, so you can’t move jars
around in load path easily
there’s no way to have it just load a .rb file; you have to have the
Service and then make that load the .rb

So I propose we do something with the jar manifest instead. For “main”
classes in jar files, you can add Main-Class: classname and classname
will be loaded and run when you java -jar the jar file. I think we
should have a JRuby-Extension: class-or-filename or similar.

Brainstorming:

JRuby-Extension: classname-or-filename

or

JRuby-Extension-Class: classname
or
JRuby-Extension-Script: something.rb

or something else?

We’ll continue to support the old way, but I think it’s time we added
something more formal and less ad-hoc.

Charlie

makki · March 1, 2012, 2:28am

Charles,

That makes perfect sense. I guess it boils down to “the niceness of ruby
is
why it is worth investing millions of lines of code of Java to put it on
the JVM”

Yoko,

Your point is well taken. I guess coming from a stronger Java background
(but with some ruby), my natural inclination is to do a lot in Java, but
perhaps it’s more appropriate to do things in Ruby. Right now I was
planning on porting a library I wrote in Ruby to Java, but I think it
might
be more appropriate to not do so…the next question becoming:

Right now, I have a project, primarily in Java, which my Ruby library
includes. I’d love it if I could have the ruby scripts just require the
jar. So for example:

ruby library pig.rb:
require ‘pig.jar’
Ruby code

ruby scripts:
require ‘pig.rb’
Ruby code

what I’d love to do is somehow compile pig.rb and put it into pig.jar
such
that I can just do require ‘pig.jar’ in a ruby script, and still get the
same library. Do you know if that is possible?

2012/2/29 Yoko H. [email protected]

makki · March 16, 2012, 6:03am

I really like the idea of changing the manifest. This is a much saner
way
to do it…but in the meantime, it’s good to know about the service
approach. I assume that the service in questions extends
BasicLibraryService, and I can just execute my .rb file in the runtime?

Thanks again. Excited about 1.7 for a lot of reasons.

2012/3/15 Charles Oliver N. [email protected]

makki · March 16, 2012, 10:24am

Maybe it is also a good time to look at formalising the API that
extensions have access to.

Currently, a JRuby extension utilises the internal JRuby API …
analogous to how CRuby extensions use its internal API. This makes
the API overly broad and quite brittle - you can’t refactor JRuby’s
internals without risking breaking some extension out there. And as
writing extensions explicitly for JRuby become more popular, it will
only get worse.

I think we could do this with a new extensions specific API, defined
as a bunch of java interfaces … as long as extensions only access
the interfaces+methods contained in the new API, they’ll be guaranteed
to work on JRuby 1.7, 1.8, 1.9, etc.

makki · March 16, 2012, 2:45pm

On Thu, Mar 15, 2012 at 11:55 PM, Charles Oliver N.
[email protected] wrote:

I hate this pattern for a couple reasons:

toplevel classes are blowful

if you have the jar in a nested path, like require ‘foo/pig.jar’,
the class it looks for is foo.PigService, so you can’t move jars
around in load path easily

there’s no way to have it just load a .rb file; you have to have the
Service and then make that load the .rb

It is magic. Implicit sucks (it will probably never happen by
accident here, but being magical it could).

JRuby-Extension-Class: classname
or
JRuby-Extension-Script: something.rb

JRuby-Require: a (same semantics as require but only search within this
Jar)

Or we could use load semantics? It would be possible for someone to
put this in a Classpath and then load/require the file already…Weird
case to be sure.

By not using extension in the name this also makes it possible to use
as a hook for things to be loaded which are not technically
extensions…What those are I don’t know, but it sounds good as I
type it

The only weird bit of it is it is a ghost require to anyone who only
expects Ruby code to be loaded as part of a require in Ruby.
Technically, it is just part of a jar require, so perhaps that is not
an issue.

or something else?

We’ll continue to support the old way, but I think it’s time we added
something more formal and less ad-hoc.

Do we ever need to support direct loading of classes over a .rb file?
I agree 1.7.0 is a perfect time to add something.

-Tom

Totals grouped by language (dominant language first):

require ‘pig.jar’

Charlie
To unsubscribe from this list, please visit:

http://xircles.codehaus.org/manage_email

–
blog: http://blog.enebo.com twitter: tom_enebo
mail: [email protected]