I’m getting an error trying to run a “runnable” war on one of our
production (Ubuntu 12.04) machines under jruby 1.7.17. We have isolated
it
just to 1.7.17, since it works OK when using 1.7.16.2, and all other
gems
and code are identical.
There must be something on that machining that’s triggering this to
happen,
since the exact same war file works on other machines. The machine was
failing while running java version 1.7.0_21, but I upgraded java
to 1.7.0_72 and it continues to fail.
Anyway, below is the stack trace, and I’d appreciate any help in
resolving
the issue. Below that I’ve listed the linked library versions, if that’s
any help.
Can you try jruby -Xnative.verbose=true -e ‘p Process.ppid’? Can you
also
tell me whether this machine has libcrypt? That was the main change to
jnr-posix around 1.7.17’s release.
Thanks for that, it certainly detected the underlying problem:
jruby -Xnative.verbose=true -e ‘p Process.ppid’
Failed to load native POSIX impl; falling back on Java impl. Stacktrace
follows.
java.lang.UnsatisfiedLinkError: libcrypt.so: cannot open shared object
file: No such file or directory
at
jnr.ffi.provider.jffi.NativeLibrary.loadNativeLibraries(NativeLibrary.java:87)
at
jnr.ffi.provider.jffi.NativeLibrary.getNativeLibraries(NativeLibrary.java:70)
at
jnr.ffi.provider.jffi.NativeLibrary.getSymbolAddress(NativeLibrary.java:49)
at
jnr.ffi.provider.jffi.NativeLibrary.findSymbolAddress(NativeLibrary.java:59)
at
jnr.ffi.provider.jffi.AsmLibraryLoader.generateInterfaceImpl(AsmLibraryLoader.java:117)
at
jnr.ffi.provider.jffi.AsmLibraryLoader.loadLibrary(AsmLibraryLoader.java:59)
at
jnr.ffi.provider.jffi.NativeLibraryLoader.loadLibrary(NativeLibraryLoader.java:43)
at jnr.ffi.LibraryLoader.load(LibraryLoader.java:265)
at jnr.ffi.Library.loadLibrary(Library.java:123)
at
jnr.posix.POSIXFactory$DefaultLibCProvider$SingletonHolder.(POSIXFactory.java:223)
at
jnr.posix.POSIXFactory$DefaultLibCProvider.getLibC(POSIXFactory.java:227)
at jnr.posix.BaseNativePOSIX.(BaseNativePOSIX.java:36)
at jnr.posix.LinuxPOSIX.(LinuxPOSIX.java:18)
at jnr.posix.POSIXFactory.loadLinuxPOSIX(POSIXFactory.java:95)
at jnr.posix.POSIXFactory.loadNativePOSIX(POSIXFactory.java:69)
at jnr.posix.POSIXFactory.loadPOSIX(POSIXFactory.java:38)
at jnr.posix.LazyPOSIX.loadPOSIX(LazyPOSIX.java:35)
at jnr.posix.LazyPOSIX.posix(LazyPOSIX.java:31)
at jnr.posix.LazyPOSIX.umask(LazyPOSIX.java:301)
at org.jruby.RubyFile.getUmaskSafe(RubyFile.java:1514)
…
NotImplementedError: getppid unsupported or native support failed to
load
But it’s not installed as libcrypt.so, since the package libc6 is
installed, but not libc6-dev, which installs and links that file. It
also
explains why this is working on our dev machines, but not on the
production-like boxes.
I’d rather not ask our clients to install development libraries on their
production machines, so is it possible to coerce jruby to use the
library
installed as .so.1 if it is found?
An unrelated question…how are you finding Savon? Did you need
compatibility with MRI Ruby? I ask because my impression was that if I
were to deal with SOAP in JRuby, I’d use a Java library since the Java
world has been using SOAP so heavily for a long time. But I never tried
it, so I don’t have any idea if that is really a wise approach.
I once tried to use Soap4R many years ago, and it was a disaster. I
heard that Savon was better, but I don’t know how much better.
So it’s picking up the library as “libcrypt.so.1” if it’s installed in
/lib
(as it is in Ubuntu 10.04), but not if it’s installed
under /lib/x86_64-linux-gnu (as it is in Ubuntu 12.04).
I can get it to load if I do either:
ln -s /lib/x86_64-linux-gnu/libcrypt-2.15.so /lib/libcrypt.so.1
or
ln -s /lib/x86_64-linux-gnu/libcrypt-2.15.so
/lib/x86_64-linux-gnu/libcrypt.so
ie link from /lib to /lib/x86_64-linux-gnu/ as libcrypt.so.1 or create
the
link as simply libcrypt.so under /lib/x86_64-linux-gnu
It feels to me like that’s not going to be the root cause, but rather
would
indicate a mis-configuration of the machine somewhere.
Is it possible to get jruby working with this configuration (eg by using
fall-backs), or do we consider this a machine configuration problem, and
fix it on our production servers?
We’ve used Savon for a couple of years now and it’s been pretty good.
There
have been a couple of hiccups with compatibility with jruby, but they
were
easily identified and quickly resolved (eg there was some code passing a
symbol when some jruby code was expecting a string).
The integration with WSDL is fairly seamless. SOAP exceptions are easy
to
handle, and processed quite naturally through ruby exceptions. I
remember
finding a couple of rough edges with some inconsistent naming inside
hashes, but haven’t dealt with them for a while, so I suspect they were
minor annoyances at the time.
On whole it’s been a good, solid choice. We’re 100% java/jruby in
production and 90% MRI in development, so libraries like this, that work
well in both environments, are important to us.
Some further investigation and googling reveals this to be a problem
with
Java on Linux, rather than specifically a jruby issue.
Java does not look at the configured library paths (configured in
/etc/ld.so.conf.d, but rather only at the environment
variable LD_LIBRARY_PATH or the Java property java.library.path. On that
machine, adding the path /lib/x86_64-linux-gnu to either of those paths
also solves the problem. We’re going to proceed with one of these
solutions
in our production environments.
Ok. This second email makes me feel a little better about putting out a
1.7.18
I have not been a sysadmin for >20 years (so you could call me really
rusty
– rusty enough to give you tetanus) but to solve these sorts of issues
I
used to just add the path to /etc/ldconfig. I think you can also set
LD_LIBRARY_PATH to include this as well if you do not have root access.
I
don’t know if Ubuntu is not exposing this directory for a particular
reason
or not but I would guess examiing what other things are in that
directory
would help figure that out. I am really really hoping something went
wrong
on your particular system and the path did not get added to ldcofig.
If someone can determine this configuration is common for Ubuntu (or
other
linuxes) we will try and figure out something which does not require
manual
intervention.
Thanks for recording what you went through. Hopefully if someone else
hits
this they will stumble across this thread.
For further note, I am using an image of Ubuntu 14.04 and Java 1.7.0u72
and
libcrypt is in the same location as yours and things are working ok. So
I
am guessing Ubuntu or Java tweaked things and fixed it in later
releases.
In the end, we clobbered the default ubuntu setup, and exposed the
libraries as described below. Note - this is quite dangerous, and if
things
go wrong it could result in a virtually unusable system.
The issue is that java is looking for libraries in /lib64 and
/usr/lib64,
but they are actually installed under /lib/x86_64-linux-gnu and
/usr/lib/x86_64-linux-gnu
First make sure there’s nothing in /lib64 other than ld-linux:
cd /
ls -Al lib64
Now, we need to use the linked location of ld-linux in the commands
below,
because we’re going to delete it, which will break pretty much
everything
until we make the symbolic link from lib64 again
