Uploading Large Files

Rails
1

I built an app that allows users to purchase video files to download,
but unfortunately media temple’s grid server isn’t letting me upload
large files. Everything goes great unless the files are over 100 mb.
I’m using attachement_fu.

Is there a service or technique that can allow me to put large files on
my server.

2

Sounds like a case for Amazon S3.

On Mar 31, 3:24 pm, Jimmy P. [email protected]

3

On 31 Mar 2008, at 21:24, Jimmy P. wrote:

I built an app that allows users to purchase video files to download,
but unfortunately media temple’s grid server isn’t letting me upload
large files. Everything goes great unless the files are over 100 mb.
I’m using attachement_fu.

Is there a service or technique that can allow me to put large files
on
my server.

FTP them, then use the Ruby File classes to read them into your
attachment_fu models. But my worry is how you’re going to serve those
files if mediatemple won’t install the xsendfile module. If you’re
going to stream those using send_file or send_data, you’re going to
get into some serious problems, no matter how they serve their rails
applications. I’d advise you to look into Amazon S3 since you seem to
be charging for your videos anyway?

Best regards

Peter De Berdt

4

Hey Peter - I thought I would just send you a direct message. You
posted some advise on this thread for me a couple months back and was
wondering if you have any further advise for me. See latest post
below.

thanks

On Jun 9, 6:59 pm, Jimmy P. [email protected]

5

FTP them, then use the Ruby File classes to read them into your
attachment_fu models. But my worry is how you’re going to serve those
files if mediatemple won’t install the xsendfile module. If you’re
going to stream those using send_file or send_data, you’re going to
get into some serious problems, no matter how they serve their rails
applications. I’d advise you to look into Amazon S3 since you seem to
be charging for your videos anyway?

Best regards

Peter De Berdt

Hey Peter - this was posted a long time ago…hopefully you are still
following this thread, but I was wondering what you mean by “you’re
going to get into some serious problems”? I’m actually using send_file
and my Ruby on Rails Container keeps terminating because it has “run out
of Ram”. This happens when the system is using the send_file function
for large files. This may have been what you previously warned me
about. Do you have any advise or techniques to overcome this issue?

I ended up doing the FTP thing as you advised, but failed to take into
consideration your send_file warning. Let me know if you have any
ideas.

thanks…

6

On 10 Jun 2008, at 01:59, Jimmy P. wrote:

going to get into some serious problems"? I’m actually using
send_file
and my Ruby on Rails Container keeps terminating because it has “run
out
of Ram”. This happens when the system is using the send_file function
for large files. This may have been what you previously warned me
about. Do you have any advise or techniques to overcome this issue?

I ended up doing the FTP thing as you advised, but failed to take into
consideration your send_file warning. Let me know if you have any
ideas.

send_file and send_data will load the entire data stream into memory
before serving it, that’s why your app goes down due to excessive
memory use.

Now, as I said, there are a couple of options you might consider:

  1. Use Apache’s or nginx’ XSendFile module to serve the files. If
    you’re using some kind of shared hosting, where you have no control
    over the actual web, then this method is gonna fall flat on its face,
    because they probably won’t want to install that module for you.
    Apache will then serve a file with a temporary url.

  2. Use an external hosting service like Amazon’s S3 to serve the
    files, where you basically offload the file serving to a service
    that’s made just for doing so. Attachment_fu should be able to help
    you out there.

Best regards

Peter De Berdt

7

thanks paul…

one last question - I’ve got over 18 gigs of video files on my media
temple server that I will need to store on the amazon s3 storage.
What’s the best way to transfer these over?

8

On 12 Jun 2008, at 22:58, Jimmy P. wrote:

one last question - I’ve got over 18 gigs of video files on my media
temple server that I will need to store on the amazon s3 storage.
What’s the best way to transfer these over?

If you have shell access to you media temple account and can run ruby
on it, using s3sync could be an option:
http://s3sync.net/wiki

Haven’t used it, but seems to fit your needs.

Best regards

Peter De Berdt