Exceptions thrown when using JRuby 1.7.0 with Amazon SDK

Ruby JRuby
1

Sorry for the non-descriptive title but I have no clue where to even
begin
looking for a cause here. I’ve only just started with JRuby, and have
attempted to use it with a short script that makes use of the AWS Ruby
SDK.
It simply attempts to list the entire contents of an S3 bucket:

Not long after starting it through JRuby, it starts spewing exceptions
from
each thread. Here is a representative example:

I am using the latest AWS SDK for Ruby gem (1.7.1), OpenJDK 7 (have also
tried 6) and have tried JRuby 1.7.0, 1.6.8 and jruby-head. All appear to
fail with the same error. Any help is appreciated.

Best Regards,
Oliver

2

Oliver -

I admit not to have a clue, but here are some questions I would ask
myself in your situation:

If I launch only one thread, would the problem persist? I do see that
the AWS gem is supposed to be thread safe here
(Forums | AWS re:Post),
but that was a while back.

I searched Amazon’s forum for help on this and only found your
message, at
Forums | AWS re:Post.
I’m posting the link here so that we can look it up.

Another question, if I had the time, would be, do I also experience
this problem using Oracle’s JDK?

  • Keith
3

On 20 November 2012 22:14, Keith B. [email protected]
wrote:

I think you copied and pasted the wrong URL here, but I recall the thread
you are talking about and I believe it is this one:

Indeed, reducing the concurrency back down to one thread seems to have
removed the problem which seems to suggest that the library is in fact
not
thread safe. Guess my little adventure with that library and JRuby ends
here.

I searched Amazon’s forum for help on this and only found your
message, at
Forums | AWS re:Post.
I’m posting the link here so that we can look it up.

Another question, if I had the time, would be, do I also experience
this problem using Oracle’s JDK?

I would probably try this option, but given the above results I don’t
think
it’s important. Correct me if this is the wrong assumption.

4

it’s unlikely to be the cause of your problem, but I’d suggest you avoid
creating all those threads and instead use java’s executor api to create
a
thread pool. check out my tip on jrubytips.com to se how:
http://jrubytips.com/posts/8-use-java-s-concurrency-library-instead-of-ruby-s-thread

since the S3 does a lot of IO you can create a thread pool with more
threads than you have cpus, try a few different numbers and see what
gives
you the best performance.

also, creating an S3 client for each thread would probably wouldn’t
hurt. I
don’t trust the thread safety of any ruby libraries.

T#

5

Good suggestions Theo, I’ll definitely give them a try. I have not as
yet
made use of any thread pooling but I can already see I’m suffering from
this decision (aside from any other performance problem). My context
switch
rate is accordingly very high, so I’ll also implement that.