Proxy without buffering

Server NGINX
1

Hello,

Large POST-request are buffered to disk, before passed to the backend.
The backend has troubles to parse the POST-data, if the requests are
huge
(some GB).

We use “haproxy” in front of “nginx”, to workaround this issue.
But this causes extra load on the server.
Is there a chance to disable the request buffering?

Thanks a lot
Markus

Posted at Nginx Forum:

2

On Mon, Feb 25, 2013 at 04:39:53AM -0500, double wrote:

Hello,

Large POST-request are buffered to disk, before passed to the backend.
The backend has troubles to parse the POST-data, if the requests are huge
(some GB).

We use “haproxy” in front of “nginx”, to workaround this issue.
But this causes extra load on the server.
Is there a chance to disable the request buffering?

Yep, it’s possible, see the other active threads for a patch from
tengine
to implement non-buffered uploads.

Also it’d be very nice to get this feature to standard nginx,
many people need it!

– Pasi

3

Thanks a lot!

Also it’d be very nice to get this feature to standard nginx,
many people need it!

100% true.
I hate this workaround via “haproxy”

Posted at Nginx Forum:

4

Hello,

I have two questions.

  1. why does backend has troubles to parse request? because parsing huge
    request make heavy load?
  2. why haproxy in front nginx can workaround the issue? I think nginx
    will
    still buffer the request to temp file

Thanks a lot

Posted at Nginx Forum:

5

Example:
Size of POST-request: 10 GB (e.g. HD video)
Upload-time: 4h
The front-server proxies the request to the upload-server.
After the upload has finished, the uploading client wants a quick “all
fine”-message (maximum 30 seconds).

If the upload-server gets the stream directly, the machine has has 4h to
parse the request (e.g. “upload-module” or “apache-fastcgi-module”). And
this does not overload the system.

Posted at Nginx Forum:

6

Hello,

  thanks for your reply. but i am still confused with the workground

about haproxy.
i know haproxy dose not buffer request, but if use haproxy in
front
of nginx,
when haproxy pass request to nginx, nginx will still buffer the
request to disk

Thanks a lot

Posted at Nginx Forum:

7

Hello,

  as you said, you use haproxy in front of nginx, that means the

request
will be proxied by haproxy first, and then be proxied by nginx,
although haproxy can proxy request immediately, but nginx can not pass
the
request on-the-fly, it will buffer request anagin. Have i made
some mistakes?

thanks a lot

Posted at Nginx Forum:

8

haproxy does not buffer the request - it proxies immediately.

Posted at Nginx Forum:

9

Machine ““front-end””:
haproxy (port 80) → proxies to “port 8080” and “machine upload”
nginx (port 8080)

Machine “upload”:
nginx (port 80) via “front-end”

Posted at Nginx Forum:

10

Hi, all

Is there any progress in this area so far? It would be great if this
did/will become a standard feature of nginx.

Posted at Nginx Forum:

11

On Thu, Jan 22, 2015 at 01:43:10AM -0500, jamesgan wrote:

Hi, all

Is there any progress in this area so far? It would be great if this
did/will become a standard feature of nginx.

+1

– Pasi