Out of memory error with file IO expected?

Ruby JRuby
1

Hello, I noticed when running this code, with 1.7.4:

100.times { Thread.new { loop { File.open(‘big’, ‘w’) do |f| f.seek
10_000_000_000; f.puts ‘a’; end}}}

after awhile, I got this failure in all threads. Perhaps the
“directBuffer” is not being freed on close or something, and could be?

http://rxr.whitequark.org/jruby/source/src/org/jruby/util/io/ChannelDescriptor.java#663

Exception in thread “RubyThread-58: (irb):1”
java.lang.OutOfMemoryError: Direct buffer memory
at java.nio.Bits.reserveMemory(Bits.java:633)
at java.nio.DirectByteBuffer.(DirectByteBuffer.java:95)
at java.nio.ByteBuffer.allocateDirect(ByteBuffer.java:288)
at
org.jruby.util.io.ChannelDescriptor.(ChannelDescriptor.java:663)
at
org.jruby.util.io.ChannelDescriptor.(ChannelDescriptor.java:205)
at
org.jruby.util.io.ChannelDescriptor.open(ChannelDescriptor.java:915)
at org.jruby.RubyFile.sysopen(RubyFile.java:1242)
at org.jruby.RubyFile.sysopenInternal(RubyFile.java:1199)
at org.jruby.RubyFile.openFile19(RubyFile.java:1150)
at org.jruby.RubyFile.initialize19(RubyFile.java:334)
at
org.jruby.RubyFile$INVOKER$i$0$2$initialize19.call(RubyFile$INVOKER$i$0$2$initialize19.gen)
at
org.jruby.runtime.callsite.CachingCallSite.callBlock(CachingCallSite.java:79)
at
org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:85)
at org.jruby.RubyClass.newInstance(RubyClass.java:876)
at org.jruby.RubyIO.open(RubyIO.java:1144)

2

I believe there was a bug reported where we were not freeing some direct
byte buffers that appears to be fixed in 1.7.5. Can you try master?

3

I would be surprised if that worked fine. :wink:

You’re opening the same file in 100 different threads, and writing to it
simultaneously. There’s no guarantee that any given thread’s operation
will not be interrupted.

You’d normally want to synchronize access to a resource like that, by
having the many threads post something to a queue, and having 1 thread
reading from the queue and doing the writes.

  • Keith