Handling File on Upload

Rails
1

Hello,

I need some help trying to figure out how to add the ability for a
user to upload a text file, and then have the application parse that
text file, and add each record as instance of a model object. I’ve
found a lot of helpful articles out there, but I’m stuck trying to
bring it all together since I’m still very new to all of this.

Here’s my view code:

index.html.erb

<% form_for :upload, :url => { :action => :upload }, :html =>
{ :multipart => true } do |f| %>

<p>Please select a file to upload:</p>
<%= f.file_field :file %>
<%= f.submit 'Upload', :disable_with => "Uploading..." %>

<% end %>

In my controller, I’ve been able to write something like this to work
with the data in my upload view:

mailing_lists_controller.rb

def upload
file = params[:upload][:file]
@data = file.read
end

so then the view looks like this:

upload.html.erb

<% @data.split("\r\n").each do |row| %>

<%= row.split("\t").join(" | ") %>

<% end %>

So, where I’m stuck is translating that into saving each row as a
record in the database. I’ve tried several different things in the
model (like creating a “process” method or using “before_save”), but
nothing has been successful thus far (probably because I’m not
implementing it correctly).

Any help would be appreciated.

Thanks,
Spencer

2

On Sun, Dec 5, 2010 at 8:54 PM, spncrgr [email protected]
wrote:

<% form_for :upload, :url => { :action => :upload }, :html =>

mailing_lists_controller.rb

def upload
file = params[:upload][:file]
@data = file.read

So here you probably want to read the file line by line and then do your
magic on each line to create the model record for the line. You may have
to
play with it - not sure if you are going to need to save the file in a
temp
location first to do this. I dealt with a similar issue and file.read on
the
uploaded file parameter gives you a string, not a file object and you
want a
file object if you want to read it line by line:

file = File.new(params[:upload][:file] # maybe you can do this… have
to
try… if not you may just need to save it to a temp location,
# perhaps
someone else has a better idea (also check params[:upload][:file] for
its
class… if the
# class is
File
then you are set.
while(line=file.gets)
m = Model.new

do your magic here

m.save
end
file.close

end

3

I recommend you to use Paperclip for files uploading.

Besides it’s other awesome features, you want to use Paperclip
Processors to play with a file after upload.
http://mdeering.com/posts/018-paperclip-processors-doing-so-much-more-with-your-attachment

To read file lines you can go for

File.readlines(@file.path).collect(&:chomp).each do |line|
… (create your models here)
end

to collect all lines.
chomp will cut end-of-line symbols, and @file it’s an object which
is given by Paperclip.
Or if you have troubles with this one, here’s another option

IO.foreach(“path/to/file.txt”) do |line|
… (create your models here)
end

4

On Dec 6, 2010, at 4:01 PM, David K. wrote:

The only thing if I recall, that kept me from using paperclip for my
purposes, is the file essentially attached to the model but does not
allow for modification and re-saving the file — am I mistaken?
Unless things have changed recently, it would be super-cool if
Paperclip would handle file mods just as a mod to any attribute of
the model… although I guess this gets away from the basic intent
of Paperclip, which for what it does do does it very nicely.

You are mistaken, and although I came late to the Paperclip party, I
can’t recall a time when it was true. You can edit the model without
modifying the image (just don’t upload another image) and everything
stays the same in the image, or you can upload a new image and it will
overwrite the previous version. It’s all managed when you save the
model that the image is attached to.

Walter

5

On Mon, Dec 6, 2010 at 3:59 PM, Walter Lee D. [email protected]
wrote:

You are mistaken, and although I came late to the Paperclip party, I can’t
recall a time when it was true. You can edit the model without modifying the
image (just don’t upload another image) and everything stays the same in the
image, or you can upload a new image and it will overwrite the previous
version. It’s all managed when you save the model that the image is attached
to.

Right, but what I wanted was to be able to load a model instance, change
the
file (say I encrypt a portion of the text) and have Paperclip update
that
file on its own when I call model#save. I am pretty sure Paperclip does
not
do this. Right, you can re-save the file but it requires manual action
beyond calling model#save.

6

On Mon, Dec 6, 2010 at 1:47 PM, Vladimir R.
[email protected]wrote:

I recommend you to use Paperclip for files uploading.
#134 Paperclip - RailsCasts

The only thing if I recall, that kept me from using paperclip for my
purposes, is the file essentially attached to the model but does not
allow
for modification and re-saving the file — am I mistaken? Unless things
have changed recently, it would be super-cool if Paperclip would handle
file
mods just as a mod to any attribute of the model… although I guess
this
gets away from the basic intent of Paperclip, which for what it does do
does
it very nicely.

7

Thanks, Vladimir and David for your replies.

I did experiment with Paperclip without much success. I look at the
post Vladimir linked to and that gave me the idea to create a custom
processor. It didn’t work though. No errors thrown, but no records
saved either :-/

I’m sure I did it wrong, but here’s the code:
http://www.pastie.org/1354053

I also experimented with taking Paperclip out of the picture and put
the code to create the records in the controller. What I get now is
an error that says “invalid attribute”. The invalid attribute
provided is the first email address of the first line of the file.

Here’s that code: http://www.pastie.org/1354053

Thanks again for any and all help. This has been my largest hurdle to
overcome in Rails thus far.

-S

8

On Dec 6, 2010, at 6:50 PM, David K. wrote:

pretty sure Paperclip does not do this. Right, you can re-save the
file but it requires manual action beyond calling model#save.

If you code your transformation within a Paperclip Processor, you can
have any number of different transformed versions. Here’s one that
extracts the text from an uploaded PDF:

#lib/paperclip_processors/text.rb
module Paperclip

Handles extracting plain text from PDF file attachments

class Text < Processor

 attr_accessor :whiny

 # Creates a Text extract from PDF
 def make
   src = @file
   dst = Tempfile.new([@basename, 'txt'].compact.join("."))
   command = <<-end_command
     "#{ File.expand_path(src.path) }"
     "#{ File.expand_path(dst.path) }"
   end_command

   begin
     success = Paperclip.run("/usr/bin/pdftotext -nopgbrk",

command.gsub(/\s+/, " "))
Rails.logger.info “Processing #{src.path} to #{dst.path} in
the text processor.”
rescue PaperclipCommandLineError
raise PaperclipError, “There was an error processing the text
for #{@basename}” if @whiny
end
dst
end
end
end

You call it from your model, like this:

#app/models/document.rb
…
has_attached_file :pdf,:styles => { :text => { :fake =>
‘variable’ } }, :processors => [:text]
…

The :fake => ‘variable’ part is just in there to get the offset
correct for the processors variable. I am not sure if it’s still
needed, but I have been doing it this way since early this summer.

Later, you can access that version of the file as you would any other
paperclip-attached model attribute. In this example, this might look
like document.pdf.url(:text). Your untouched original will always be
at document.pdf.url(:original).

Walter

9

sorry…that was the wrong URL for the second pastie. Here’s the
right one: http://www.pastie.org/1354138

10

I was finally able to get this to work. I ditched Paperclip and went
with putting the code in the controller. Taking it in baby steps, I
was able to work out this code for the controller:

def upload
((params[:upload][:file]).read).strip.split("\r\n").each do |line|
email, account_number, sub_number, eid, premium_code, keycode,
order_date, description = line.split("\t")

  new_record = MailingList.new(:email => email, :account_number =>

account_number, :sub_number => sub_number,
:eid => eid, :premium_code =>
premium_code, :keycode => keycode,
:order_date =>
order_date, :description => description)
new_record.save
end

redirect_to :action => index

end

Thanks again to everyone. Ultimately it was all of your replies that
helped me piece it together.

Thanks!
Spencer

11

Hi spncrgr :smiley:

I am trying to do something similar…Is there a possibility that you
have
this project on Github? (or somewhere, where i can view the code?)

Thank you in advance

George.

12

On 3 December 2014 at 10:41, George S. [email protected] wrote:

Hi spncrgr :smiley:

I am trying to do something similar…Is there a possibility that you have
this project on Github? (or somewhere, where i can view the code?)

I rather doubt whether code from four years ago is likely to still be
the best solution.

Colin