I’m hoping somebody can give me some direction on how to approach this
problem. We have various projects with data acquisition in the field.
These projects produce time series data of variables such as
temperature, and power usage. I’m using javascript to plot the data.
Currently I have a form in which the user would submit the project
name, the type of data (60 minute, 1 minute, hourly, etc.) and the
names of the variables requested. This generates an ajax request to
find and parse a csv file to get the variables and send them back to
the browser to plot.
The csv files are updated daily. New projects are created often. Each
project has several types of data with many variables and many rows.
I’d like to have the data all in a database, possibly ActiveRecord,
taking advantage of validations and such. I’ve thought of two
approaches so far:
-
Creating a unique new table for every project and for every type of
data (e.g. data_project1_60minute). This results in creating new
classes for every table as well, which doesn’t seem optimal. There is
also the possibility of using the “set_table_name” feature of
activerecord, but I wonder if this is the best way. -
Creating one massive table for every type of data (e.g.
data_60minute, with column names project1_var1, project1_var2,
project2_var1, etc.). Then adding columns whenever a new project is
added. This also doesn’t seem great since one data’s 60minute has time
differences between the others. There is also event driven data which
collects at high frequency for short periods of time, which also
doesn’t fit this model.
Really it seems like I want some of the features of ActiveRecord, but
I don’t really want the object relational mapping, and it gets in the
way by requiring me to make a bunch of new classes.
Thanks for any advice that someone can give.