Too many open files with imap connections

Ruby JRuby
1

Running into an issue with jruby throwing too many open files when
opening imap connections. I am not sure, but I have a hunch it’s an
issue specifically with ssl connections. Able to reproduce with this
script:

require ‘net/imap’
10000.times.each do |i|
puts “connection #{i}”
imap = Net::IMAP.new(‘imap.gmail.com’, 993, true)
imap.disconnect
end

Depending on OS settings, eventually java raises:

OpenSSL::SSL::SSLError: Too many open files
from /home/ubuntu/.rvm/rubies/jruby-1.5.1/lib/ruby/1.8/net/imap.rb:
904:in initialize' from (irb):4:in new’
from (irb):4

I am running

jruby 1.5.1
java version “1.6.0_18”
OpenJDK Runtime Environment (IcedTea6 1.8) (6b18-1.8-0ubuntu1)
OpenJDK 64-Bit Server VM (build 14.0-b16, mixed mode)

Thx.

-karl

To unsubscribe from this list, please visit:

http://xircles.codehaus.org/manage_email
2

On Aug 15, 2010, at 9:15 PM, Karl B. wrote:

OpenJDK 64-Bit Server VM (build 14.0-b16, mixed mode)

Thx.

-karl

Karl,

This should not be a surprise. All computers have finite resources, and
if you exhaust them, you’ll run into problems.

Specifically, the “problem” (in quotation marks) manifests itself with
MRI as well.

Cheers.

To unsubscribe from this list, please visit:

http://xircles.codehaus.org/manage_email
3

beside the limit of resources net-ldap indeed does not close the
connections which is patch on svn head:

http://rubyforge.org/tracker/index.php?func=detail&aid=24178&group_id=143&atid=633

regards Kristian

On Mon, Aug 16, 2010 at 9:13 AM, Hirotsugu A. [email protected]
wrote:

end
jruby 1.5.1
Karl,

To unsubscribe from this list, please visit:

http://xircles.codehaus.org/manage_email
4

Tried this with ruby enterprise 1.8.7 and I did not run into the
issue. Will also give it a go with MRI later today.

In regards to it not being a surprise, I’m not sure I understand what
you are saying. Since we are opening and immediately closing the
connection, I would think things would be getting cleaned up.

thx

-karl

On Aug 15, 2010, at 11:43 PM, Hirotsugu A. wrote:

puts “connection #{i}”
from (irb):4
-karl

Cheers.

To unsubscribe from this list, please visit:

http://xircles.codehaus.org/manage_email

To unsubscribe from this list, please visit:

http://xircles.codehaus.org/manage_email
5

For kicks, I modified your script as follows:

10000.times.each do |i|
puts “connection #{i}”
imap = Net::IMAP.new(‘imap.gmail.com’, 993, true)
imap.disconnect
lsof > imap-#{i}.txt if (i % 1000) == 0
end

With my Mac OS 10.6, I could not reproduce the problem. Did you see this
problem with a Sun JVM ?

However, the lsof output shows a lot of pipes left open. I can attach
them
to a bug if you open one.

Thanks,

Antoine

6

Installed the sun jdk on ubuntu and ran into the same issue. Will try
on OSX later on this morning.

Thanks!

7

It took at least 10 minutes. In my script, I captured the lsof output
every
1000 connections. The result shows exponentially more imap connections
opened.

I fiddled with my mac quite a bit and I might have managed to change the
soft limit of open files, or I was just lucky not to crash. In any case,
this is a bug for sure, and I can show some lsof output to prove it.

Thanks,

Antoine

8

Should I open up a jruby ticket for this?

Also, any ideas of ways i can work around this in the time being?

Thanks!

-karl

9

Please do. Not sure you can work around it for now.

10

Ran into the same issue on OSX:

Mac OSX 10.5.8
Jruby 1.5.1
java version “1.6.0_17”
Java™ SE Runtime Environment (build 1.6.0_17-b04-248-9M3125)
Java HotSpot™ 64-Bit Server VM (build 14.3-b01-101, mixed mode)

On ruby 1.8.7 I did not have the issue: ruby 1.8.7 (2010-01-10
patchlevel 249) [i686-darwin9.8.0])

Antoine, I am surprised that you did not run into the issue on OSX.
My only guess is that I am not yet running OSX 10.6 as i have been
lazy about upgrading. How long did you let it run for? When you run
“lsof -p | wc -l” do you see the open file descriptors actually
go back down? I will get one of my colleagues with OSX 10.6 to try
and reproduce.

thx.

11

Created a ticket:

http://jira.codehaus.org/browse/JRUBY-5015

thx.

-karl