Cext build failure

ratchetcat · August 16, 2010, 1:54am

On 16 August 2010 08:39, Jon [email protected] wrote:

The symbols might not be exported correctly
for windows, when jruby-cext.dll is built, it probably needs an
additional linker flag to export all symbols.

Agreed. Using the graphical http://www.dependencywalker.com/ shows jruby-cext.dll only exporting symbols (ordinals 1-40) like “JNI_OnLoad” and “Java_org_jruby_cext_Native_newRString” and “Java_org_jruby_cext_Native_getTrue” but none of the jruby implementations of the MRI symbols.

Ok, that gives me a clue. All of those use JNIEXPORT in their
declaration+definition, which most likely explicitly exports those
symbols - a quick&dirty fix would be to do the same for all the rb_*
and jruby_* functions we export - prefix at least the definition’s
with JNIEXPORT and see if that works.

I think you can also have mingw’s linker read a list of symbols to
export from a file, which may be another alternative.

From what you’ve seen so far do you agree that RbConfigLibrary.java will likely need to be updated as well as cext’s Makefile?

Sounds like it. I don’t think tweaking those specifically for
win32/64 will be a problem.

To unsubscribe from this list, please visit:

http://xircles.codehaus.org/manage_email

ratchetcat · August 16, 2010, 4:31am

From what you’ve seen so far do you agree that RbConfigLibrary.java will likely need to be updated as well as cext’s Makefile?

Sounds like it. I don’t think tweaking those specifically for
win32/64 will be a problem.

You may not need to prefix with JNIEXPORT as an explicit
–export-all-symbols in the cext Makefile fixed a lot of things.

We’re sooo close…the attached patch (needs scrubbed) allows me to
build jruby_cext.dll and rdiscount in a normal windows cmd.exe shell
(not the MSYS shell as it appears my build.xml edits cause problems)
with a C:\DevKit\cc.bat

…but I get a NoMethodError when trying to use rdiscount

C:\Users\Jon\Documents\JavaDev\jruby>ant clean cext

C:\Users\Jon\Documents\CDev\rdiscount>jruby -S gem install
rdiscount-1.6.5.jm1.gem
JRuby limited openssl loaded. http://jruby.org/openssl
gem install jruby-openssl for full support.
Temporarily enhancing PATH to include DevKit…
Building native extensions. This could take a while…
Successfully installed rdiscount-1.6.5.jm1
1 gem installed

C:\Users\Jon\Documents\CDev\rdiscount>jruby -rubygems -e “require
‘rdiscount’; puts RDiscount.new(‘hello’).to_html”
-e:1: undefined method `to_html’ for #<RDiscount:0x37165f
@text=“hello”> (NoMethodError)

ratchetcat · August 16, 2010, 6:01am

On 16 August 2010 12:30, Jon [email protected] wrote:

Successfully installed rdiscount-1.6.5.jm1
1 gem installed

C:\Users\Jon\Documents\CDev\rdiscount>jruby -rubygems -e “require ‘rdiscount’; puts RDiscount.new(‘hello’).to_html”
-e:1: undefined method `to_html’ for #<RDiscount:0x37165f @text=“hello”> (NoMethodError)

Hmmm. Unless tim has removed the debug, there should be a line like
“calling init(0x12345678)” output to stdout or stderr. this could
indicate that the native module is not getting loaded.

It might be an idea to try some of the example modules in jruby
(wmeissner-jruby-cext/examples I think, if they’re still there). They
have an alternative, simpler build system, which might make debugging
the init/loading sequence a bit easier.

To unsubscribe from this list, please visit:

http://xircles.codehaus.org/manage_email

ratchetcat · August 15, 2010, 11:31pm

On 16 August 2010 06:24, Jon [email protected] wrote:

went away when I forced linkage against both c:\ruby187\bin\msvcrt-ruby18.dll and c:\ruby191\bin\msvcrt-ruby191.dll I’m sure this isn’t how cext is supposed to work, but it tells me we’re missing a link against the

Linking against anything from MRI is really, really bad. jruby-cext
provides the jruby implemention of those symbols, and mixing the two
will make jruby asplode. The symbols might not be exported correctly
for windows, when jruby-cext.dll is built, it probably needs an
additional linker flag to export all symbols.

To unsubscribe from this list, please visit:

http://xircles.codehaus.org/manage_email

ratchetcat · August 16, 2010, 11:47am

Hi

On Aug 16, 2010, at 6:01 AM, Wayne M. wrote:

Temporarily enhancing PATH to include DevKit…
indicate that the native module is not getting loaded.
It is not removed, rdiscount master does not currently load on JRuby,
see timfel/rdiscount for a patch (I’ve already sent a pull request to
Ryan).
It seems that MRI loads “rdiscount.bundle” (on Mac) even when ‘require
“rdiscount.so”’ is in the code. We might have to mirror this behavior,
as I personally don’t like just forcing an .so extension on the cext
libraries (as in Jon’s patch).

From what you’ve seen so far do you agree that RbConfigLibrary.java will likely need to be updated as well as cext’s Makefile?

Sounds like it. I don’t think tweaking those specifically for
win32/64 will be a problem.

I will apply parts of your patch today.

-Tim

To unsubscribe from this list, please visit:

http://xircles.codehaus.org/manage_email

ratchetcat · August 16, 2010, 3:16pm

Hmmm. Unless tim has removed the debug, there should be a line like
“calling init(0x12345678)” output to stdout or stderr. this could
indicate that the native module is not getting loaded.

It is not removed, rdiscount master does not currently load on JRuby, see timfel/rdiscount for a patch (I’ve already sent a pull request to Ryan).
It seems that MRI loads “rdiscount.bundle” (on Mac) even when ‘require “rdiscount.so”’ is in the code. We might have to mirror this behavior, as I personally don’t like just forcing an .so extension on the cext libraries (as in Jon’s patch).

Does the patch incorrectly determine the platform type before deciding
on an extension? My intention was to change from .dll to .so only on
windows and not touch the Mac stuff. And the .so change on windows was
because I thought I remembered MRI 1.9 needed it’s extensions named
.so…this may be completely wrong and irrelevant to JRuby.

I will apply parts of your patch today.

After seeing that jruby-cext.dll exports 652 symbols of which ~45% start
with either rb, ruby, or jruby I think that --export-all-symbols gives
too much surface area and Wayne’s initial idea of explicitly exporting
symbols is the way to go.

FYI, I’m moving back to focusing on polishing thingw to work better with
the DevKit as I want to release the new DevKit ASAP and I want it to
have JRuby support. Specifically, (a) remove the need for cc.bat and
cc.sh stubs as CC should be overridden by operating_system.rb and
devkit.rb similar to how PATH is temporarily updated, and (b) cleanup
build.xml so that it works in both standard cmd.exe and MSYS shells.
That said, don’t hesitate to ping me if you’d like another set of
testing eyes on Win7.

Jon

To unsubscribe from this list, please visit:

http://xircles.codehaus.org/manage_email

ratchetcat · August 16, 2010, 3:36pm

On Aug 16, 2010, at 3:20 PM, Jon wrote:

C:\Users\Jon\Documents\CDev\rdiscount>jruby -rubygems -e “require ‘rdiscount’; puts RDiscount.new(‘hello’).to_html”
-e:1: undefined method `to_html’ for #<RDiscount:0x37165f @text=“hello”> (NoMethodError)

Hmmm. Unless tim has removed the debug, there should be a line like
“calling init(0x12345678)” output to stdout or stderr. this could
indicate that the native module is not getting loaded.

Would the successfull rdiscount require and the RDiscount.new indicate the native module is getting loaded? The error message makes it look like the instance is being created but then something is going wrong in the method lookup logic.

No, the native module in RDiscount is loaded after the RDiscount class
is defined in Ruby, the native codes basically only adds the to_html
method. If that isn’t there, the native extension hasn’t been loaded.

To unsubscribe from this list, please visit:

http://xircles.codehaus.org/manage_email

ratchetcat · August 18, 2010, 12:18am

Just a short note, gotta catch some sleep:
I pushed parts of Jon’s patch today and added explicit dllexports for
the build and defined dllimports in the ruby.h for extensions using it.
I was then able to completely build cext support and run Thin on my
Windows machine here. I haven’t gotten the specs to work, but I already
know the fix for that, just didn’t have the time, yet.
Jon, could you test? I have e.g. native mongrel and rfuzz with passing
specs here, so those should definitely run. You could also try curb,
eventmachine, racc, ruby-pg which I have mostly working here (mostly
some spec failures here and there)
-Tim

To unsubscribe from this list, please visit:

http://xircles.codehaus.org/manage_email

ratchetcat · August 16, 2010, 3:20pm

C:\Users\Jon\Documents\CDev\rdiscount>jruby -rubygems -e “require ‘rdiscount’; puts RDiscount.new(‘hello’).to_html”
-e:1: undefined method `to_html’ for #<RDiscount:0x37165f @text=“hello”> (NoMethodError)

Hmmm. Unless tim has removed the debug, there should be a line like
“calling init(0x12345678)” output to stdout or stderr. this could
indicate that the native module is not getting loaded.

Would the successfull rdiscount require and the RDiscount.new indicate
the native module is getting loaded? The error message makes it look
like the instance is being created but then something is going wrong in
the method lookup logic.

To unsubscribe from this list, please visit:

http://xircles.codehaus.org/manage_email

ratchetcat · August 18, 2010, 3:39am

On 18 August 2010 11:03, Jon [email protected] wrote:

should -fexceptions be removed from RbConfigLibrary.java?

If its used when compiling the c extensions, then definitely not. Its
needed so C++ exceptions (which is how rb_raise() is implemented) get
propagated thru the C extension code. Without -fexceptions, as soon
as an exception is raised, the jvm terminates … which is usually not
what one wants.

To unsubscribe from this list, please visit:

http://xircles.codehaus.org/manage_email

ratchetcat · August 18, 2010, 3:04am

I pushed parts of Jon’s patch today and added explicit dllexports for the build and defined dllimports in the ruby.h for extensions using it. I was then able to completely build cext support and run Thin on my Windows machine here. I haven’t gotten the specs to work, but I already know the fix for that, just didn’t have the time, yet.

cext built without error and exports 331 symbols rather than 652 with
–export-all-symbols

Jon, could you test? I have e.g. native mongrel and rfuzz with passing specs here, so those should definitely run. You could also try curb, eventmachine, racc, ruby-pg which I have mostly working here (mostly some spec failures here and there)
-Tim

RedCloth fails due to missing rb_obj_clone symbol
EventMachine fails due to unrecognized -EHs and -GR options
should -fexceptions be removed from RbConfigLibrary.java?
attached patch removes -mimpure-text (Solaris only, now Win32?) and
makes CC usage a bit more flexible.
FYI, I still haven’t found a way to remove cc.bat and cc (shell
script) from DevKit so that the Makefile’s built from
RbConfigLibrary.java can have their “CC=cc” overridden.

=== FAILURE DETAILS ===

C:\Users\Jon\Documents\CDev\rdiscount>gem install RedCloth
–platform=ruby
JRuby limited openssl loaded. http://jruby.org/openssl
gem install jruby-openssl for full support.
Temporarily enhancing PATH to include DevKit…
Building native extensions. This could take a while…
ERROR: Error installing RedCloth:
ERROR: Failed to build gem native extension.

C:/Users/Jon/Documents/JavaDev/jruby/bin/jruby.exe extconf.rb
WARNING: JRuby does not support native extensions or the `mkmf’ library
very well.
Check http://kenai.com/projects/jruby/pages/Home for alternatives.
C:/Users/Jon/Documents/JavaDev/jruby/lib/ruby/1.8/mkmf.rb:31 warning:
already initiali
zed constant RUBY_PLATFORM
creating Makefile

make
cc -I. -IC:/Users/Jon/Documents/JavaDev/jruby/lib/native/include
-IC:/Users/Jon/Docum
ents/JavaDev/jruby/lib/native/include/ruby -I. -fno-omit-frame-pointer
-fno-strict-al
iasing -fexceptions -O2 -m32 -march=native -mtune=native -c
redcloth_attributes.c
cc -I. -IC:/Users/Jon/Documents/JavaDev/jruby/lib/native/include
-IC:/Users/Jon/Docum
ents/JavaDev/jruby/lib/native/include/ruby -I. -fno-omit-frame-pointer
-fno-strict-al
iasing -fexceptions -O2 -m32 -march=native -mtune=native -c
redcloth_inline.c
cc -I. -IC:/Users/Jon/Documents/JavaDev/jruby/lib/native/include
-IC:/Users/Jon/Docum
ents/JavaDev/jruby/lib/native/include/ruby -I. -fno-omit-frame-pointer
-fno-strict-al
iasing -fexceptions -O2 -m32 -march=native -mtune=native -c
redcloth_scan.c
cc -shared -o redcloth_scan.dll redcloth_attributes.o
redcloth_inline.o redcloth_sca
n.o -L"." -L"/Users/Jon/Documents/JavaDev/jruby/lib"
-LC:/Users/Jon/Documents/JavaDev/
jruby/lib/native/i386-Windows -ljruby-cext
-Wl,–enable-auto-image-base,–enable-auto-
import -m32 -march=native -mtune=native
redcloth_scan.o:redcloth_scan.c:(.text+0xa33b): undefined reference to
`rb_obj_clone’
collect2: ld returned 1 exit status
make: *** [redcloth_scan.dll] Error 1

C:\Users\Jon\Documents\CDev\rdiscount>gem install eventmachine
–platform=ruby
JRuby limited openssl loaded. http://jruby.org/openssl
gem install jruby-openssl for full support.
Temporarily enhancing PATH to include DevKit…
Building native extensions. This could take a while…
ERROR: Error installing eventmachine:
ERROR: Failed to build gem native extension.

C:/Users/Jon/Documents/JavaDev/jruby/bin/jruby.exe extconf.rb
WARNING: JRuby does not support native extensions or the `mkmf’ library
very well.
Check http://kenai.com/projects/jruby/pages/Home for alternatives.
C:/Users/Jon/Documents/JavaDev/jruby/lib/ruby/1.8/mkmf.rb:31 warning:
already initiali
zed constant RUBY_PLATFORM
checking for rb_trap_immediate in ruby.h,rubysig.h… no
checking for rb_thread_blocking_region()… conftest.c:2:0: fatal error:
when writing
output to : Broken pipe
compilation terminated.
checking for inotify_init() in sys/inotify.h… no
checking for __NR_inotify_init in sys/syscall.h… no
checking for writev() in sys/uio.h… no
checking for rb_thread_check_ints()… no
checking for rb_time_new()… conftest.c:2:0: fatal error: when writing
output to : In
valid argument
compilation terminated.
checking for windows.h… yes
checking for winsock.h… yes
checking for main() in -lkernel32… yes
checking for main() in -lrpcrt4… yes
checking for main() in -lgdi32… yes
checking for main() in -lssleay32… no
creating Makefile

make
g++ -I. -I.
-IC:/Users/Jon/Documents/JavaDev/jruby/lib/native/include/ruby -I.
-DBUILD
_FOR_RUBY -DHAVE_RB_THREAD_BLOCKING_REGION -DHAVE_TBR -DHAVE_RB_TIME_NEW
-DOS_WIN32 -D
HAVE_WINDOWS_H -DHAVE_WINSOCK_H -EHs -GR -DWITHOUT_SSL
-fno-omit-frame-pointer -fno-s
trict-aliasing -fexceptions -m32 -march=native -mtune=native -c
binder.cpp
g++.exe: unrecognized option ‘-EHs’
g++.exe: unrecognized option ‘-GR’
g++ -I. -I.
-IC:/Users/Jon/Documents/JavaDev/jruby/lib/native/include/ruby -I.
-DBUILD
_FOR_RUBY -DHAVE_RB_THREAD_BLOCKING_REGION -DHAVE_TBR -DHAVE_RB_TIME_NEW
-DOS_WIN32 -D
HAVE_WINDOWS_H -DHAVE_WINSOCK_H -EHs -GR -DWITHOUT_SSL
-fno-omit-frame-pointer -fno-s
trict-aliasing -fexceptions -m32 -march=native -mtune=native -c
cmain.cpp
cmain.cpp: In function ‘int evma_send_file_data_to_connection(long
unsigned int, const
char*)’:
cmain.cpp:752:20: error: ‘fstat’ was not declared in this scope
g++.exe: unrecognized option ‘-EHs’
g++.exe: unrecognized option ‘-GR’
make: *** [cmain.o] Error 1

ratchetcat · August 18, 2010, 5:19pm

Jon, could you test? I have e.g. native mongrel and rfuzz with passing specs here, so those should definitely run. You could also try curb, eventmachine, racc, ruby-pg which I have mostly working here (mostly some spec failures here and there)

Based upon failures attempting to install curb, I think JRuby’s RubyGems
may be causing some of the problems I’m seeing when trying to override
CC and will cause problems if one tries to install a native gem and
needs to explicitly point to headers/libs.

The following works with the MRI 1.9.2 branch and MRI trunk (1.9.3).
Notice that in jruby’s call to extconf.rb, none of the build args given
to “gem install” are passed.

Can you replicate what I’m seeing with a clean PATH like?

C:\Users\Jon\Documents>echo %PATH%
c:\Users\Jon\Documents\JavaDev\jruby\bin;C:\Windows\system32;C:\Windows;C:\Windows\System32\Wbem

C:\Users\Jon\Documents>gem install curb --platform=ruby –
–with-curl-lib=“C:/gnuwin32/curl/bin”
–with-curl-include=“C:/gnuwin32/curl/include”
JRuby limited openssl loaded. http://jruby.org/openssl
gem install jruby-openssl for full support.
Temporarily enhancing PATH to include DevKit…
Building native extensions. This could take a while…
ERROR: Error installing curb:
ERROR: Failed to build gem native extension.

c:/Users/Jon/Documents/JavaDev/jruby/bin/jruby.exe extconf.rb
WARNING: JRuby does not support native extensions or the `mkmf’ library
very well.
Check http://kenai.com/projects/jruby/pages/Home for alternatives.
c:/Users/Jon/Documents/JavaDev/jruby/lib/ruby/1.8/mkmf.rb:31 warning:
already initialized constant RUBY_PLATFORM
" -fno-omit-frame-pointer -fno-strict-aliasing -fexceptions"
checking for curl-config… no
checking for main() in -lcurl… no
*** extconf.rb failed ***

To unsubscribe from this list, please visit:

http://xircles.codehaus.org/manage_email

ratchetcat · August 18, 2010, 3:30pm

FYI, I still haven’t found a way to remove cc.bat and cc (shell script) from DevKit so that the Makefile’s built from RbConfigLibrary.java can have their “CC=cc” overridden.

Something’s very wrong, either in JRuby’s mkmf.rb or my understanding.

When I run, using either ‘gcc’ or ‘gcc.exe’ for CC…

C:\Users\Jon\Documents\CDev\rdiscount>set CC=gcc.exe && gem install
rdiscount-1.6.5.jm1.gem --platform=ruby – --environment-overrides

…this should override any CC in the extensions Makefile created from
running extconf.rb. Not so.

As “make -p | grep CC” shows make uses “CC = cc” by default, I tried
deleting

http://github.com/jruby/jruby/blob/master/src/org/jruby/libraries/RbConfigLibrary.java#L289

in hopes the extension Makefile simply wouldn’t define CC similar to
how CXX is currently undefined. Unfortunately the generated Makefile
had “CC =” and I couldn’t override it from the environment.

I’m loathe to start spelunking through JRuby 1.8’s custom mkmf.rb…any
other ideas or something I’m overlooking?

Jon

To unsubscribe from this list, please visit:

http://xircles.codehaus.org/manage_email

ratchetcat · August 18, 2010, 11:04pm

On Aug 18, 2010, at 3:30 PM, Jon wrote:

As “make -p | grep CC” shows make uses “CC = cc” by default, I tried deleting

http://github.com/jruby/jruby/blob/master/src/org/jruby/libraries/RbConfigLibrary.java#L289

in hopes the extension Makefile simply wouldn’t define CC similar to how CXX is currently undefined. Unfortunately the generated Makefile had “CC =” and I couldn’t override it from the environment.

I’m loathe to start spelunking through JRuby 1.8’s custom mkmf.rb…any other ideas or something I’m overlooking?

It’s not actually that ‘custom’… the modifications are only at the
very top for working around things which work differently in RbConfig,
in “have_func” and in the removal of the dependency on defines.h and
ruby.h

The line
http://github.com/jruby/jruby/blob/cext/lib/ruby/1.8/mkmf.rb#L1348 is
the reason why the CC variable is always set in the Makefile (see
http://github.com/ruby/ruby/blob/trunk/lib/mkmf.rb#L1507 where it’s done
in the same way, only they also include the CXX compiler).

Based upon failures attempting to install curb, I think JRuby’s RubyGems may be causing some of the problems I’m seeing when trying to override CC and will cause problems if one tries to install a native gem and needs to explicitly point to headers/libs.

Okay, there are two problems here, first rubygems isn’t passing the
paramters to the extconf.rb and secondly the paramter
‘environment-overrides’ is ignored entirely. Do you know where MRI
handles the latter? I couldn’t find a reference to it in mkmf.rb.
Additionally, I have seen a few extconf.rb’s which handle this
themselves ( ENV[“CC”] || CONFIG[“CC”] ).

-Tim

To unsubscribe from this list, please visit:

http://xircles.codehaus.org/manage_email

ratchetcat · August 18, 2010, 11:30pm

I’ve also played around with adding “RbConfig::CONFIG[‘CC’] = ‘gcc’” to the DevKit-installed operating_system.rb with no luck. Also tried sqlite3-ruby/ext/sqlite3/extconf.rb at master · sparklemotion/sqlite3-ruby · GitHub with no luck.

In mkmf the Makefile values are taken from RbConfig::CONFIG hash, not
MAKEFILE_CONFIG hash. That’s why I “merge!” MAKEFILE_CONFIG over CONFIG
in the header of mkmf.rb. This is why setting RbConfig::CONFIG[‘CC’] =
‘gcc’ in the operating_system.rb won’t do, as the mkmf.rb is loaded
afterwards and just overrides this. In the sqlite3/extconf.rb it’s the
opposite: MAKEFILE_CONFIG is modified, but that is ignored here, as it
has been previously force-merged into CONFIG for this execution and is
ignored further down.

-Tim

To unsubscribe from this list, please visit:

http://xircles.codehaus.org/manage_email

ratchetcat · August 18, 2010, 11:18pm

It’s not actually that ‘custom’… the modifications are only at the very top for working around things which work differently in RbConfig, in “have_func” and in the removal of the dependency on defines.h and ruby.h

The line http://github.com/jruby/jruby/blob/cext/lib/ruby/1.8/mkmf.rb#L1348 is the reason why the CC variable is always set in the Makefile (see http://github.com/ruby/ruby/blob/trunk/lib/mkmf.rb#L1507 where it’s done in the same way, only they also include the CXX compiler).

Thanks for the pointers, I’ll start investigating.

Okay, there are two problems here, first rubygems isn’t passing the paramters to the extconf.rb and secondly the paramter ‘environment-overrides’ is ignored entirely. Do you know where MRI handles the latter? I couldn’t find a reference to it in mkmf.rb. Additionally, I have seen a few extconf.rb’s which handle this themselves ( ENV[“CC”] || CONFIG[“CC”] ).

I think those params are gathered up by rubygems and then made part of a
“jruby extconf.rb --my-build-option --another-build-option” backtick
call. Haven’t had a chance to track it down and verify yet.

I’ve also played around with adding “RbConfig::CONFIG[‘CC’] = ‘gcc’” to
the DevKit-installed operating_system.rb with no luck. Also tried

github.com

sparklemotion/sqlite3-ruby/blob/master/ext/sqlite3/extconf.rb#L7


      
          require "mkmf"
          require "yaml"
          
          
module Sqlite3
            module ExtConf
              ENV_ALLOWLIST = ["CC", "CFLAGS", "LDFLAGS", "LIBS", "CPPFLAGS", "LT_SYS_LIBRARY_PATH", "CPP"]
          
          
    class << self
                def configure
                  configure_cross_compiler
          
          
        if system_libraries?
                    message "Building sqlite3-ruby using system #{libname}.\n"
                    configure_system_libraries
                  else
                    message "Building sqlite3-ruby using packaged sqlite3.\n"
                    configure_packaged_libraries

with no luck.

I want to check whether manually running “jruby extconf.rb --blah --blah
–blah” from inside the curb ext folder works. That should tell us
whether JRuby’s mkmf.rb is guilty of ignoring its parameters.

Jon

To unsubscribe from this list, please visit:

http://xircles.codehaus.org/manage_email

ratchetcat · August 19, 2010, 5:18pm

Building this morning with bf3e11a using “ant clean cext” gives me the
following failure:

 [exec] gcc -DNDEBUG -O2 -fno-omit-frame-pointer

-fno-strict-aliasing -W -Wa
ll -Wno-unused -Wno-parentheses -Werror -Wundef
-I"/c/Users/Jon/Documents/JavaDe
v/jruby/cext/src/…/…//build"
-I"/c/Users/Jon/Documents/JavaDev/jruby/cext/src"
-I"/c/Users/Jon/Documents/JavaDev/jruby/cext/src/…/…//build"/jni
-I"/c/Users/
Jon/Documents/JavaDev/jruby/cext/src"/include
-I"/c/Users/Jon/Documents/JavaDev/
jruby/cext/src"/include/ruby -I"C:\Program
Files\Java\jdk1.6.0_21/include" -I"C
:\Program Files\Java\jdk1.6.0_21/include/win32" -D_REENTRANT
-D_LARGEFILE64_SOU
RCE -D_GNU_SOURCE -march=native -mtune=native -D_JNI_IMPLEMENTATION_
-DRUBY_DLLS
PEC="__declspec(dllexport)" -c
/c/Users/Jon/Documents/JavaDev/jruby/cext/src/st.
c -o /c/Users/Jon/Documents/JavaDev/jruby/cext/src/…/…//build/st.o
[exec] In file included from
c:/Users/Jon/Documents/JavaDev/jruby/cext/src/
include/ruby.h:2:0,
[exec] from
c:/Users/Jon/Documents/JavaDev/jruby/cext/src/
st.c:5:
[exec]
c:/Users/Jon/Documents/JavaDev/jruby/cext/src/include/ruby/ruby.h:90
8:33: error: external linkage required for symbol
‘rb_thread_stop_timer_thread’
because of ‘dllexport’ attribute
[exec]
c:/Users/Jon/Documents/JavaDev/jruby/cext/src/include/ruby/ruby.h:90
9:33: error: external linkage required for symbol
‘rb_thread_start_timer_thread’
because of ‘dllexport’ attribute
[exec]
c:/Users/Jon/Documents/JavaDev/jruby/cext/src/include/ruby/ruby.h:91
0:33: error: external linkage required for symbol ‘rb_thread_stop_timer’
because
of ‘dllexport’ attribute
[exec]
c:/Users/Jon/Documents/JavaDev/jruby/cext/src/include/ruby/ruby.h:91
1:33: error: external linkage required for symbol
‘rb_thread_start_timer’ becaus
e of ‘dllexport’ attribute
[exec] make: ***
[/c/Users/Jon/Documents/JavaDev/jruby/cext/src/…/…//buil
d/st.o] Error 1

To unsubscribe from this list, please visit:

http://xircles.codehaus.org/manage_email

ratchetcat · August 19, 2010, 3:30pm

In mkmf the Makefile values are taken from RbConfig::CONFIG hash, not MAKEFILE_CONFIG hash. That’s why I “merge!” MAKEFILE_CONFIG over CONFIG in the header of mkmf.rb. This is why setting RbConfig::CONFIG[‘CC’] = ‘gcc’ in the operating_system.rb won’t do, as the mkmf.rb is loaded afterwards and just overrides this. In the sqlite3/extconf.rb it’s the opposite: MAKEFILE_CONFIG is modified, but that is ignored here, as it has been previously force-merged into CONFIG for this execution and is ignored further down.

I’m hoping we find a robust mod to the current behavior that will allow
either RbConfig::CONFIG[‘CC’] or ENV[‘CC’] overrides in
operating_system.rb’s pre_install hook and
lib/ruby/site_ruby/shared/devkit.rb to work similar to how I override
ENV[‘PATH’] to bring in the DevKit in only very specific situations,
i.e. ‘gem install’ and ‘jruby -rdevkit extconf.rb’

Things are going to be crazy busy at work for me starting today through
next week but I’ll reply with any new info I might unearth.

Did you discover anything on the missing build args issue with the
“jruby extconf.rb” call? I’m going to try the “jruby -rdevkit
extconf.rb --with-curl-lib --with-curl-include” with curb to see if
mkmf.rb is really ignoring them.

Jon

To unsubscribe from this list, please visit:

http://xircles.codehaus.org/manage_email

ratchetcat · August 21, 2010, 6:09pm

FYI, a few tweaks to RbConfigLibrary.java (attached) and the following
mods to the DevKit helper files and I’m able to override CC correctly
for “jruby -rdevkit extconf.rb” cases.

There’s still something in JRuby/RubyGems that’s preventing CC overrides
for the “gem install my_ext --platform=ruby” cases and I think we still
have the problem of the build options given to “gem install” not being
passed to the “jruby extconf.rb” call.

Jon

FILE: lib/ruby/site_ruby/shared/devkit.rb

enable RubyInstaller DevKit usage as a vendorable helper library

unless ENV[‘PATH’].include?(‘C:\DevKit\mingw\bin’) then
require ‘rbconfig’
puts ‘Temporarily enhancing PATH to include DevKit…’
ENV[‘PATH’] = ‘C:\DevKit\bin;C:\DevKit\mingw\bin;’ + ENV[‘PATH’]
RbConfig::MAKEFILE_CONFIG[‘CC’] = ‘gcc’
end

FILE: lib/ruby/site_ruby/1.8/rubygems/defaults/operating_system.rb

override ‘gem install’ to enable RubyInstaller DevKit usage

Gem.pre_install do |i|
require ‘rbconfig’
unless ENV[‘PATH’].include?(‘C:\DevKit\mingw\bin’) then
puts ‘Temporarily enhancing PATH to include DevKit…’
ENV[‘PATH’] = ‘C:\DevKit\bin;C:\DevKit\mingw\bin;’ +
ENV[‘PATH’]
end
RbConfig::MAKEFILE_CONFIG[‘CC’] = ‘gcc’
end

ratchetcat · August 19, 2010, 11:05pm

“ant clean cext” now works for me with 9a285a3.

Running the following indicates JRuby’s mkmf.rb handles the extconf.rb
args OK and something in jruby (rubygems?) is not sending the args from
the “gem install” command. Didn’t have time to look into the CC
override issue today.

FYI, I just released the new DevKit at
Downloads with usage info at
http://wiki.github.com/oneclick/rubyinstaller/development-kit and I’ll
release another version once we’re both happy with the mods needed for
things to work flawlessly with JRuby cext.

Jon

C:\Users\Jon\Documents\JavaDev\jruby\lib\ruby\gems\1.8\gems\curb-0.7.8\ext>jruby
-rdevkit extconf.rb --with-curl-lib=“C:/gnuwin32/curl/bin”
–with-curl-include=“C:/gnuwin32/curl/include”
Temporarily enhancing PATH to include DevKit…
WARNING: JRuby does not support native extensions or the `mkmf’ library
very well.
Check http://kenai.com/projects/jruby/pages/Home for alternatives.
C:/Users/Jon/Documents/JavaDev/jruby/lib/ruby/1.8/mkmf.rb:31 warning:
already initiali
zed constant RUBY_PLATFORM
" -fno-omit-frame-pointer -fno-strict-aliasing -fexceptions"
checking for curl-config… no
checking for main() in -lcurl… yes
checking for curl/curl.h… yes
checking for curlinfo_redirect_time… yes
checking for curlinfo_response_code… yes
checking for curlinfo_filetime… yes
checking for curlinfo_redirect_count… yes
checking for curlinfo_os_errno… yes
checking for curlinfo_num_connects… yes
checking for curlinfo_ftp_entry_path… yes
checking for curl_version_ssl… yes
checking for curl_version_libz… yes
checking for curl_version_ntlm… yes
checking for curl_version_gssnegotiate… yes
checking for curl_version_debug… yes
checking for curl_version_asynchdns… yes
checking for curl_version_spnego… yes
checking for curl_version_largefile… yes
checking for curl_version_idn… yes
checking for curl_version_sspi… yes
checking for curl_version_conv… yes
checking for curlproxy_http… yes
checking for curlproxy_socks4… yes
checking for curlproxy_socks5… yes
checking for curlauth_basic… yes
checking for curlauth_digest… yes
checking for curlauth_gssnegotiate… yes
checking for curlauth_ntlm… yes
checking for curlauth_anysafe… yes
checking for curlauth_any… yes
checking for curle_tftp_notfound… yes
checking for curle_tftp_perm… yes
checking for curle_tftp_diskfull… yes
checking for curle_tftp_illegal… yes
checking for curle_tftp_unknownid… yes
checking for curle_tftp_exists… yes
checking for curle_tftp_nosuchuser… yes
checking for curle_send_fail_rewind… yes
checking for curle_ssl_engine_initfailed… yes
checking for curle_login_denied… yes
checking for curlmopt_maxconnects… yes
checking for curle_conv_failed… yes
checking for curle_conv_reqd… yes
checking for curle_ssl_cacert_badfile… yes
checking for curle_remote_file_not_found… yes
checking for curle_ssh… yes
checking for curle_ssl_shutdown_failed… yes
checking for curle_again… yes
checking for curle_ssl_crl_badfile… yes
checking for curle_ssl_issuer_error… yes
checking for curlopt_username… yes
checking for curlopt_password… yes
checking for curlinfo_primary_ip… yes
checking for curlauth_digest_ie… yes
checking for curlftpmethod_multicwd… yes
checking for curlftpmethod_nocwd… yes
checking for curlftpmethod_singlecwd… yes
checking for curlm_bad_socket… yes
checking for curlm_unknown_option… yes
checking for curl_multi_timeout()… yes
checking for curl_multi_fdset()… yes
checking for curl_multi_perform()… yes
checking for Ruby 1.9 Hash… no
checking for Ruby 1.9 st.h… no
checking for curl_easy_escape… yes
creating curb_config.h
creating Makefile

To unsubscribe from this list, please visit:

http://xircles.codehaus.org/manage_email

Cext build failure

No, the native module in RDiscount is loaded after the RDiscount class is defined in Ruby, the native codes basically only adds the to_html method. If that isn’t there, the native extension hasn’t been loaded.

-Tim

FILE: lib/ruby/site_ruby/shared/devkit.rb

enable RubyInstaller DevKit usage as a vendorable helper library

FILE: lib/ruby/site_ruby/1.8/rubygems/defaults/operating_system.rb

override ‘gem install’ to enable RubyInstaller DevKit usage

No, the native module in RDiscount is loaded after the RDiscount class
is defined in Ruby, the native codes basically only adds the to_html
method. If that isn’t there, the native extension hasn’t been loaded.