FasterCSV, Gruff and Large Data Import

Ruby
1

Hi all,

I’ve got a CSV file which contains 18k items that I need to import to
graph. I’m using FasterCSV and Gruff to import and graph the data.

Obviously I’m going about this wrong as it’s overloading the Array
class. Is there another way around this?

http://pastie.org/412741

Kind Regards,

Dan

2

On Mar 10, 2009, at 9:03 AM, Dan W. [dbw] wrote:

I’ve got a CSV file which contains 18k items that I need to import to
graph. I’m using FasterCSV and Gruff to import and graph the data.

Obviously I’m going about this wrong as it’s overloading the Array
class. Is there another way around this?

http://pastie.org/412741

Well, I think you probably need to trim the data down a bit. Could
you maybe build up some averages over time periods and feed just those
averages to Gruff?

James Edward G. II

3

On Mar 10, 2009, at 10:36 AM, James G. wrote:

http://pastie.org/412741

Well, I think you probably need to trim the data down a bit. Could
you maybe build up some averages over time periods and feed just
those averages to Gruff?

James Edward G. II

What kind of graph? How big is the graph? Think about what kind of
resolution is meaningful – the default is only 800 pixels wide and you
can’t graph sub-pixel values :wink:

-Rob

Rob B. http://agileconsultingllc.com
[email protected]

4

70:in `each_with_index’

line 20’s where the array gets pushed into the graph.

Can you show us your code? Looks like your array of data may itself
contain
arrays – have you tried flattening the array before giving it to Gruff?

5

On Mar 10, 2009, at 9:57 AM, Dan W. [dbw] wrote:

On a second look the error I’m getting is a gruff related one.

c:/ruby/lib/ruby/gems/1.8/gems/gruff-0.3.4/lib/gruff/base.rb:943:in
larger_than _max?': undefined method>’ for [“1\t8877”]:Array (NoMethodError)
from

Gruff expects an Array of values to graph. It looks like you might be
feeding it an Array of Arrays and even those values look fishy.

I recommend simplifying the problem. First, try just reading the
first five lines of your data file and graphing just that.

When you have that working, you can expand the data and see how things
go.

James Edward G. II

6

Hi again,

As suggested I’ve changed tact. It now looks like gruff doesn’t like the
array created by FasterCSV, but I don’t know why!

If I create an array manually then feed it into gruff, i.e.

a = [1,2,3,4,5]
g.data(“X”, a)

it works absolutely fine. However if I’ve put the same data into a csv
file, then read it in using either:

a = fastercsv.read(“r.csv”)

or

FasterCSV.foreach(“r.csv”) do |row|
a << row
end

If I puts all three they all print out as
1
2
3
4
5

And both errors given by the array when I use fasterCSV are:

c:/ruby/lib/ruby/gems/1.8/gems/gruff-0.3.4/lib/gruff/base.rb:943:in
larger_than _max?': undefined method>’ for [“1”, “2”, “3”, “4”, “5”]:Array
(NoMethodError)

    from

c:/ruby/lib/ruby/gems/1.8/gems/gruff-0.3.4/lib/gruff/base.rb:453:in
data' from c:/ruby/lib/ruby/gems/1.8/gems/fastercsv-1.4.0/lib/faster_csv.rb:15 70:ineach_with_index’
from
c:/ruby/lib/ruby/gems/1.8/gems/gruff-0.3.4/lib/gruff/base.rb:443:in
each' from c:/ruby/lib/ruby/gems/1.8/gems/gruff-0.3.4/lib/gruff/base.rb:443:ineach_with_index’
from
c:/ruby/lib/ruby/gems/1.8/gems/gruff-0.3.4/lib/gruff/base.rb:443:in
`data’
from main.rb:27

am I missing something spectacular?
Thanks,
Dan

7

FasterCSV.foreach(“r.csv”) do |row|
a << row
end

Looks like you’re feeding the whole CSV table into Gruff all at once.
Try
something like this:

idx = 1
FasterCSV.foreach(“r.csv”) do |row|
g.data(“Series #{idx}”, row)
idx += 1
end

8

However if I feed the fastercsv data into an array then into gruff it
should work, as feeding a whole array that I’ve written out by hand
into gruff works. If you see my logic here, it should work, no?

Dan

9

Not really, well I could but I’m trying to compare it with doing it in a
similar program such as GNU Plot, plus there are sometimes
irregularities in the data which I’d like to show if possible.

On a second look the error I’m getting is a gruff related one.

c:/ruby/lib/ruby/gems/1.8/gems/gruff-0.3.4/lib/gruff/base.rb:943:in
larger_than _max?': undefined method>’ for [“1\t8877”]:Array (NoMethodError)
from
c:/ruby/lib/ruby/gems/1.8/gems/gruff-0.3.4/lib/gruff/base.rb:453:in
data' from c:/ruby/lib/ruby/gems/1.8/gems/fastercsv-1.4.0/lib/faster_csv.rb:15 70:ineach_with_index’
from
c:/ruby/lib/ruby/gems/1.8/gems/gruff-0.3.4/lib/gruff/base.rb:443:in
each' from c:/ruby/lib/ruby/gems/1.8/gems/gruff-0.3.4/lib/gruff/base.rb:443:ineach_with_index’
from
c:/ruby/lib/ruby/gems/1.8/gems/gruff-0.3.4/lib/gruff/base.rb:443:in
`data’
from main.rb:20

line 20’s where the array gets pushed into the graph.

Dan

10

2009/3/11 Dan W. [dbw] [email protected]

However if I feed the fastercsv data into an array then into gruff it
should work, as feeding a whole array that I’ve written out by hand
into gruff works. If you see my logic here, it should work, no?

No, because reading a CSV produces a table, i.e. an array of arrays,
something like:

FasterCSV.read(“r.csv”) #=> [[1,2,3,4], [5,6,7,8], …]

Gruff expects a flat array of data for each data series, so you need to
loop
over each row in the table and feed them to Gruff one by one.