Re: POST with huge HTTP body

Server Mongrel
1

A while ago I wrote a plugin to limit uploads in Mongrel. Mongrel does
save uploads to a tempfile on disk - if the upload is bigger than about
12 KB - using a Ruby TempFile-object. (which stores in the $TMPDIR, /tmp
on most systems).

The request is handed over to Rails after it’s fully received by
Mongrel.

I’m not sure if this “saving to disk” works the same with chunked
uploads (uploads without a Content-Length header).

Maybe my plugin can help you: http://slasaus.netsend.nl/articles/show/7
(warning: it works, but is not very elegant).

Gl,

Tim

----- Original Message ----
From: Daniel B. [email protected]
To: [email protected]
Sent: Monday, October 15, 2007 2:00:20 PM
Subject: [Mongrel] POST with huge HTTP body

Hi,

I found the following page describing how to stream data from the
server to the client:

http://api.rubyonrails.com/classes/ActionController/Streaming.html

I want to do the opposite, streaming data from the client to the
server,
letting the controller saving the data while it’s being received.
In particular, I want to upload large files to the RailsDav
(http://www.liverail.net/railsdav) plugin, without having to store
the entire file in memory during the operation.

This seems to be a Mongrel problem, so I started looking at this
plugin:

http://mongrel.rubyforge.org/docs/upload_progress.html

How do I get Mongrel to store the incoming POST data in a tempfile,
passing a handler to that file as the raw_post field in the Rails
request?
Is it even possible?
How is the “request” parameter to the process() method related to the
“request” object in the Rails controllers?

/Daniel

Mongrel-users mailing list
[email protected]
http://rubyforge.org/mailman/listinfo/mongrel-users

  ____________________________________________________________________________________

Shape Yahoo! in your own image. Join our Network Research Panel today!
http://surveylink.yahoo.com/gmrs/yahoo_panel_invite.asp?a=7

2

Thanks for the replies.

The fact that Mongrel shouldn’t be the first one to get the POST/PUT
data
was a good point that I hadn’t thought of. However, even when talking
directly to Mongrel, the memory consumption of the application increases
until it crashes if too much data is sent. In my case, I was sending a
500MB
file from a Webdav client. At some point all of it was loaded into RAM,
which
doesn’t work.

Are there any more productive alternatives than writing the entire thing
from scratch in C? Please? :slight_smile:

/Daniel

3

On Mon, 15 Oct 2007 16:10:06 +0200
“Daniel B.” [email protected] wrote:

from scratch in C? Please? :slight_smile:
Daniel,

Mongrel is a library. Not a server. I just wrote a server with the
library. Look past mongrel_rails and into the heart of the RDoc. You
will find your answer there.

(Probably somewhere in the place where it mentions handlers and
filters.)

–
Zed A. Shaw