Simple_xlsx_writer doesn't work

dubstep · September 21, 2011, 9:13pm

I wanted to try using simple_xlsx_writer to generate Excel files but the
gem won’t install with jRuby. The problem is the gem references fast_xs
which is a C extension (fast_xs is used for escaping text)

Has anyone figured out a way to get simple_xlsx_writer to work with
jRuby?

ken_e · September 21, 2011, 9:26pm

On Sep 21, 2011, at 2:13 PM, Ken E. wrote:

I wanted to try using simple_xlsx_writer to generate Excel files but the
gem won’t install with jRuby. The problem is the gem references fast_xs
which is a C extension (fast_xs is used for escaping text)

Has anyone figured out a way to get simple_xlsx_writer to work with
jRuby?

You didn’t specifically mention this, but I assume you are running this
on Windows. JRuby does support most C extensions now, but it needs the
correct development environment to build them (unless they are provided
in binary form already).

You provided no error output from when you tried to install the
extension, so it’s hard to say the root cause.

To compile the extension on Windows, you need to make sure have DevKit
installed. Go to rubyinstaller.org and search for DevKit, install it on
your system, and try again. If you still have problems, write back here
with more information.

cr

ken_e · September 21, 2011, 11:19pm

On Wed, Sep 21, 2011 at 9:13 PM, Ken E. [email protected] wrote:

I wanted to try using simple_xlsx_writer to generate Excel files but the
gem won’t install with jRuby. The problem is the gem references fast_xs
which is a C extension (fast_xs is used for escaping text)

Has anyone figured out a way to get simple_xlsx_writer to work with
jRuby?

if you really need to stick to simple_xlsx_writer because you’ve code
that was working in ruby and not jruby, it might just be indeed a need
to recompile it.

but since this is jruby, you could instead of simple_xlsx_writer use a
java library to create excel files, with poi apache.

if you’re interested in jruby/poi, I can share some code for you to
create xls or xlsx files. let me know

–
Christian

ken_e · September 22, 2011, 12:37am

Chuck R. wrote in post #1023168:

On Sep 21, 2011, at 2:13 PM, Ken E. wrote:

I wanted to try using simple_xlsx_writer to generate Excel files but the
gem won’t install with jRuby. The problem is the gem references fast_xs
which is a C extension (fast_xs is used for escaping text)

Has anyone figured out a way to get simple_xlsx_writer to work with
jRuby?

You didn’t specifically mention this, but I assume you are running this
on Windows. JRuby does support most C extensions now, but it needs the
correct development environment to build them (unless they are provided
in binary form already).

You provided no error output from when you tried to install the
extension, so it’s hard to say the root cause.

To compile the extension on Windows, you need to make sure have DevKit
installed. Go to rubyinstaller.org and search for DevKit, install it on
your system, and try again. If you still have problems, write back here
with more information.

cr

Yes, I’m running jRuby on Windows. I haven’t had the need to compile
anything with Windows as most gems work as-is with jRuby.

Anyway, I will give it a try.

ken_e · September 22, 2011, 12:45am

Christian MICHON wrote in post #1023186:

On Wed, Sep 21, 2011 at 9:13 PM, Ken E. [email protected] wrote:

I wanted to try using simple_xlsx_writer to generate Excel files but the
gem won’t install with jRuby. The problem is the gem references fast_xs
which is a C extension (fast_xs is used for escaping text)

Has anyone figured out a way to get simple_xlsx_writer to work with
jRuby?

if you really need to stick to simple_xlsx_writer because you’ve code
that was working in ruby and not jruby, it might just be indeed a need
to recompile it.

but since this is jruby, you could instead of simple_xlsx_writer use a
java library to create excel files, with poi apache.

if you’re interested in jruby/poi, I can share some code for you to
create xls or xlsx files. let me know

–
Christian

Thanks, I used POI before and while its feature packed, we ran into
memory issues and a heap dump analysis pointed to POI as source of a
memory leak. I moved our code to the spreadsheet gem and now finding it
struggles with a large number of rows. simple_xlsx_writer solves this
problem by dumping rows straight to an XML stream however its pretty
basic in function.

ken_e · September 22, 2011, 4:25pm

Christian MICHON wrote in post #1023222:

On Thu, Sep 22, 2011 at 12:45 AM, Ken E. [email protected] wrote:

if you really need to stick to simple_xlsx_writer because you’ve code
Christian

Thanks, I used POI before and while its feature packed, we ran into
memory issues and a heap dump analysis pointed to POI as source of a
memory leak. I moved our code to the spreadsheet gem and now finding it
struggles with a large number of rows. simple_xlsx_writer solves this
problem by dumping rows straight to an XML stream however its pretty
basic in function.

I never ran into memory issues with poi (I did with jxl).

2 questions:

was it with a recent version of poi and not a beta ? (we’re using 3.7)

what is the size of the expected excel output file and average
number of rows ?

We’re using it with thousands of rows with no problems, hence my
surprise.

–
Christian

I used the most current version as of last March. Our users generate
lots of spreadsheets, some small and some large. The larger ones tend to
be 10-15k rows with lots of columns. We started running into memory
issues on our server (which is still a problem since our JVM is maxed at
1.2gb on a 32bit Windows server) which caused us to suspect a memory
leak.

I can’t say for sure there was really a memory leak but when more than
half of our heap is filled with objects all linked to POI, the analyzer
suggested a leak.

Moving to the spreadsheet gem seemed to make things better but memory
usage is still high. We’re in the process of moving to a 64bit server,
but in the meantime, I’m seeing if a different spreadsheet approach will
help.

ken_e · September 22, 2011, 4:29pm

Chuck R. wrote in post #1023168:

On Sep 21, 2011, at 2:13 PM, Ken E. wrote:

I wanted to try using simple_xlsx_writer to generate Excel files but the
gem won’t install with jRuby. The problem is the gem references fast_xs
which is a C extension (fast_xs is used for escaping text)

Has anyone figured out a way to get simple_xlsx_writer to work with
jRuby?

You didn’t specifically mention this, but I assume you are running this
on Windows. JRuby does support most C extensions now, but it needs the
correct development environment to build them (unless they are provided
in binary form already).

You provided no error output from when you tried to install the
extension, so it’s hard to say the root cause.

To compile the extension on Windows, you need to make sure have DevKit
installed. Go to rubyinstaller.org and search for DevKit, install it on
your system, and try again. If you still have problems, write back here
with more information.

cr

No luck with DevKit. My development machine is 64bit Windows and the
devkit only works for 32bit. Reinstalling jRuby with 32 bit is an option
I guess…

ken_e · September 22, 2011, 7:38am

On Thu, Sep 22, 2011 at 12:45 AM, Ken E. [email protected] wrote:

if you really need to stick to simple_xlsx_writer because you’ve code
Christian

Thanks, I used POI before and while its feature packed, we ran into
memory issues and a heap dump analysis pointed to POI as source of a
memory leak. I moved our code to the spreadsheet gem and now finding it
struggles with a large number of rows. simple_xlsx_writer solves this
problem by dumping rows straight to an XML stream however its pretty
basic in function.

I never ran into memory issues with poi (I did with jxl).

2 questions:

was it with a recent version of poi and not a beta ? (we’re using 3.7)
what is the size of the expected excel output file and average
number of rows ?

We’re using it with thousands of rows with no problems, hence my
surprise.

–
Christian

ken_e · September 22, 2011, 4:43pm

On Sep 22, 2011, at 9:29 AM, Ken E. wrote:

You didn’t specifically mention this, but I assume you are running this
with more information.

cr

No luck with DevKit. My development machine is 64bit Windows and the
devkit only works for 32bit. Reinstalling jRuby with 32 bit is an option
I guess…

Every response gives us a few more important details… knowing you are
on windows x64 is interesting. I use Windows7 x64 every day with a
32-bit JVM for doing COM work. The 64-bit JVM, while giving you larger
max heaps as you know, cannot connect to a 32-bit COM interface
directly. You could use a COM proxy to get around this limitation but
performance will be a fraction of what it is with a direct link.

Here are instructions from Microsoft on how to isolate a 32-bit COM DLL
to its own process so that a 64-bit process (like the JVM) can
communicate with it via COM.

BTW, DevKit installs and runs just fine on Windows 64-bit but as you
likely discovered it only generates 32-bit binaries right now. That’s
being worked on by folks on the RubyInstaller team who are far smarter
than me.

Let us know how you eventually solve this.

cr