Building JRuby 1.5.0 fails

Ruby JRuby
1

Hi folks,

As you may know we build JRuby from sources as part of the NetBeans
build process. After upgrading to JRuby 1.5 building JRuby fails on
Gentoo, which prevents providing an NB ebuild for Gentoo. Below is the
stacktrace, anyone seen this before? This appears to be fully
reproducible on the ebuild maintaner’s (in Cc) setup when running the
dist-bin target. (Of course the error may not be related to Gentoo at
all.)

[…]
/home/fordfrog/src/gentoo/tmp/jruby-1.5.0/build_lib/jffi-x86_64-Windows.jar
into
/home/fordfrog/src/gentoo/tmp/jruby-1.5.0/dist/jruby-bin-1.5.0/lib/native
[echo] Running rake install_dist_gems
[echo] compile=OFF, threshold=20, objectspace=true threadpool=false
reflection=false
[java]
/home/fordfrog/src/gentoo/tmp/jruby-1.5.0/lib/ruby/site_ruby/1.8/rubygems.rb:838:in
searcher': Mutex relocking by same thread (ThreadError) [java] from /home/fordfrog/src/gentoo/tmp/jruby-1.5.0/lib/ruby/site_ruby/1.8/rubygems/custom_require.rb:33:inrequire’
[java] from
/home/fordfrog/src/gentoo/tmp/jruby-1.5.0/lib/ruby/site_ruby/1.8/rubygems/defaults/jruby.rb:92:in
spec_directories_from_classpath' [java] from /home/fordfrog/src/gentoo/tmp/jruby-1.5.0/lib/ruby/site_ruby/1.8/rubygems/defaults/jruby.rb:82:ininstalled_spec_directories’
[java] from
/home/fordfrog/src/gentoo/tmp/jruby-1.5.0/lib/ruby/site_ruby/1.8/rubygems/source_index.rb:58:in
from_installed_gems' [java] from /home/fordfrog/src/gentoo/tmp/jruby-1.5.0/lib/ruby/site_ruby/1.8/rubygems.rb:881:insource_index’
[java] from
/home/fordfrog/src/gentoo/tmp/jruby-1.5.0/lib/ruby/site_ruby/1.8/rubygems/gem_path_searcher.rb:81:in
init_gemspecs' [java] from /home/fordfrog/src/gentoo/tmp/jruby-1.5.0/lib/ruby/site_ruby/1.8/rubygems/gem_path_searcher.rb:13:ininitialize’
[java] from
/home/fordfrog/src/gentoo/tmp/jruby-1.5.0/lib/ruby/site_ruby/1.8/rubygems.rb:839:in
new' [java] from /home/fordfrog/src/gentoo/tmp/jruby-1.5.0/lib/ruby/site_ruby/1.8/rubygems.rb:839:insearcher’
[java] from
/home/fordfrog/src/gentoo/tmp/jruby-1.5.0/lib/ruby/site_ruby/1.8/rubygems.rb:838:in
searcher' [java] from /home/fordfrog/src/gentoo/tmp/jruby-1.5.0/lib/ruby/site_ruby/1.8/rubygems.rb:478:infind_files’
[java] from
/home/fordfrog/src/gentoo/tmp/jruby-1.5.0/lib/ruby/site_ruby/1.8/rubygems.rb:1103
[java] from
/home/fordfrog/src/gentoo/tmp/jruby-1.5.0/lib/ruby/site_ruby/1.8/rubygems.rb:9:in
`require’
[java] from
/home/fordfrog/src/gentoo/tmp/jruby-1.5.0/bin/rake:9

BUILD FAILED
/home/fordfrog/src/gentoo/tmp/jruby-1.5.0/build.xml:1292: The following
error
occurred while executing this line:
/home/fordfrog/src/gentoo/tmp/jruby-1.5.0/build.xml:1043: Java returned:
1

Total time: 27 seconds

Thanks,
Erno

To unsubscribe from this list, please visit:

http://xircles.codehaus.org/manage_email
2

Hi Erno,

That is a peculiar error, I haven’t seen it before during a build.
What are the hardware characteristics on the build system?
http://ci.jruby.org/ runs on a Gentoo-based EC2 instance, and we have
no problems. The only thing that comes to mind is some odd
thread-scheduling bug that we just haven’t seen before.

/Nick

On Fri, May 28, 2010 at 2:11 AM, Erno M. [email protected]
wrote:

/home/fordfrog/src/gentoo/tmp/jruby-1.5.0/build_lib/jffi-x86_64-Windows.jar
`require’
/home/fordfrog/src/gentoo/tmp/jruby-1.5.0/lib/ruby/site_ruby/1.8/rubygems.rb:881:in
[java] from
[java] from
Total time: 27 seconds

To unsubscribe from this list, please visit:

http://xircles.codehaus.org/manage_email
3

Can you figure out the bits (scripts, etc) being used for the build?
I’ve never seen this before.

Is this just doing the provided Gentoo ebuild for JRuby?

On Fri, May 28, 2010 at 2:11 AM, Erno M. [email protected]
wrote:

/home/fordfrog/src/gentoo/tmp/jruby-1.5.0/build_lib/jffi-x86_64-Windows.jar
`require’
/home/fordfrog/src/gentoo/tmp/jruby-1.5.0/lib/ruby/site_ruby/1.8/rubygems.rb:881:in
  [java]   from
  [java]   from
Total time: 27 seconds

To unsubscribe from this list, please visit:

http://xircles.codehaus.org/manage_email
4

Thanks for the reply Nick, here’s some more info on the setup where this
happens:

OS: linux (gentoo)

$ java -version
java version “1.6.0_20”
Java™ SE Runtime Environment (build 1.6.0_20-b02)
Java HotSpot™ 64-Bit Server VM (build 16.3-b01, mixed mode)

$ equery list ant

  • Searching for ant …
    [IP-] [ ] dev-java/ant-1.8.1:0

I’ll provide more details on the HW as I get it. The error was
originally encountered when building NB, which in turn builds JRuby with
a patched build.xml, but this is reproducible with the unmodified JRuby
sources with the following steps:

$ wget
http://jruby.org.s3.amazonaws.com/downloads/1.5.0/jruby-src-1.5.0.zip
$ unzip jruby-src-1.5.0.zip
$ cd jruby-1.5.0
$ ant install-dist-gems && ant dist-bin

I’ve asked the original reporter to try building a few more times to
make sure this really isn’t a random threading related issue.

Thanks,
Erno

Nick S. wrote:

On Fri, May 28, 2010 at 2:11 AM, Erno M. [email protected] wrote:

[…]
/home/fordfrog/src/gentoo/tmp/jruby-1.5.0/lib/ruby/site_ruby/1.8/rubygems/custom_require.rb:33:in
[java] from
`new’
/home/fordfrog/src/gentoo/tmp/jruby-1.5.0/lib/ruby/site_ruby/1.8/rubygems.rb:1103

To unsubscribe from this list, please visit:

http://xircles.codehaus.org/manage_email
5

hi,

Charles Oliver N. <headius headius.com> writes:

Can you figure out the bits (scripts, etc) being used for the build?
I’ve never seen this before.

Is this just doing the provided Gentoo ebuild for JRuby?

i’m the one who reported the problem with building jruby while building
netbeans. on gentoo, we do not have jruby 1.5.0 ebuild yet so i did not
have
even opportunity to check the ebuild to see what works. what i did was
just
plain build using sources as described by Erno in his post.

i’m not on the list so if you need more info from me, either add me to
copy or
send me mail directly.

miroslav

To unsubscribe from this list, please visit:

http://xircles.codehaus.org/manage_email
6

Also, open a bug. We’re probably going to proceed to spin 1.5.1 today
since we haven’t been able to reproduce this and since it seems like a
more “raw” issue with loading JRuby on your particular platform
configuration. We’ll work with you to get it building, and deal with
it if there’s changes required.

On Fri, Jun 4, 2010 at 12:55 PM, Charles Oliver N.
[email protected] wrote:

In bin/rake:
 def require(path) # :doc:

OS: linux (gentoo)
I’ll provide more details on the HW as I get it. The error was originally
sure this really isn’t a random threading related issue.

That is a peculiar error, I haven’t seen it before during a build.

error may not be related to Gentoo at all.)

 [java]   from
`source_index’
/home/fordfrog/src/gentoo/tmp/jruby-1.5.0/lib/ruby/site_ruby/1.8/rubygems.rb:839:in

Thanks,

To unsubscribe from this list, please visit:

 http://xircles.codehaus.org/manage_email

To unsubscribe from this list, please visit:

http://xircles.codehaus.org/manage_email
7

Thanks a lot guys! I filed JRUBY-4852 for this, I’ll work with Miroslav
(fordfrog) to add logging to jruby.rb and provide more info in the issue
then.

Erno

Charles Oliver N. wrote:

looks like the gems.rb file is getting loaded twice, causing the
require ‘rubygems’
rescue LoadError => load_error
then matching whatever that regexp matches, so the Gem.searcher call
I tried them on Linux, both with Ant 1.7.1 and 1.8.1, even in a

$ java -version
build.xml, but this is reproducible with the unmodified JRuby sources with
Thanks,

http://ci.jruby.org/ runs on a Gentoo-based EC2 instance, and we have

[…]
`searcher’: Mutex relocking by same thread (ThreadError)
/home/fordfrog/src/gentoo/tmp/jruby-1.5.0/lib/ruby/site_ruby/1.8/rubygems/defaults/jruby.rb:82:in

[java] from
`find_files’
/home/fordfrog/src/gentoo/tmp/jruby-1.5.0/build.xml:1292: The following

To unsubscribe from this list, please visit:

To unsubscribe from this list, please visit:

http://xircles.codehaus.org/manage_email
8

So Tom and I have done some exploration. Looking at your backtrace, it
looks like the gems.rb file is getting loaded twice, causing the
searcher mutex to be re-entered…and Ruby Mutex is not reentrant.
Since the only way to cause a file to load twice would be to either
explicitly load it or to require it with different paths, I’d expect
that something is causing the second require to be against a different
path than the first one.

Looking at the code in both places that require gems.rb:

In bin/rake:

require ‘rubygems’

In lib/ruby/site_ruby/1.8/rubygems/defaults/jruby.rb:

require ‘jruby/util’

Because RubyGems is live, this goes back through custom_require in
RubyGems to find this file, eventually hitting this logic:

def require(path) # :doc:
gem_original_require path
rescue LoadError => load_error
if load_error.message =~ /#{Regexp.escape path}\z/ and
spec = Gem.searcher.find(path) then
Gem.activate(spec.name, “= #{spec.version}”)
gem_original_require path
else
raise load_error
end
end

So it would seem that somehow jruby/util is producing a LoadError and
then matching whatever that regexp matches, so the Gem.searcher call
runs when it’s already in-flight. A fragility in RubyGems, certainly,
but the root problem is jruby/util not loading properly.

I think the path forward is to figure out why you’re getting a load
error there, probably by logging it in the code above. jruby/util is a
builtin library, and it shouldn’t be possible for it to fail to load
through normal means; there has to be something else blowing up during
its load.

FWIW, Tom tried various combinations of your build process on OS X and
I tried them on Linux, both with Ant 1.7.1 and 1.8.1, even in a
symlinked directory…and nothing broke. There’s something else going
on.

On Tue, Jun 1, 2010 at 6:38 AM, Erno M. [email protected]
wrote:

$ unzip jruby-src-1.5.0.zip

/Nick

prevents providing an NB ebuild for Gentoo. Below is the stacktrace,
/home/fordfrog/src/gentoo/tmp/jruby-1.5.0/dist/jruby-bin-1.5.0/lib/native
`require’
/home/fordfrog/src/gentoo/tmp/jruby-1.5.0/lib/ruby/site_ruby/1.8/rubygems/source_index.rb:58:in

 [java]   from
 [java]   from
1
 http://xircles.codehaus.org/manage_email

To unsubscribe from this list, please visit:

 http://xircles.codehaus.org/manage_email

To unsubscribe from this list, please visit:

http://xircles.codehaus.org/manage_email