Mongrel errors overload Nginx's error.log and server's CPU

Rails Deployment
1

Hello,

I’m considering to replace our production server Apache with Enginx.
Behind them there are a dozen of mongrel_rails instances. Some of the
features that I like very much in Nginx are - fair load balancing for
proxy upstreams and the posibility to define servers as backup.

This is my current configuration:

http {
include /etc/nginx/mime.types;
default_type application/octet-stream;

log_format  main  '$remote_addr - $remote_user [$time_local]

$request ’
'“$status” $body_bytes_sent “$http_referer” ’
‘“$http_user_agent” “$http_x_forwarded_for”’;

access_log  /var/log/nginx/access.log  main;

sendfile        on;
keepalive_timeout  65;

gzip  on;
gzip_min_length  1100;
gzip_buffers     4 8k;
gzip_types       text/html text/plain text/xml text/javascript

text/css

charset utf-8;
upstream_fair_shm_size 64k;

upstream mongrel {
    server    127.0.0.1:8000;
    server    127.0.0.1:8001;
}
server { ...
location / {
    proxy_set_header    X-Real-IP  $remote_addr;
    proxy_set_header    X-Forwarded-For $proxy_add_x_forwarded_for;
    proxy_set_header    Host $http_host;
    proxy_redirect  false;

    if (-f $request_filename/index.html) {
       rewrite (.*) $1/index.html break;
    }
    if (-f $request_filename.html) {
       rewrite (.*) $1.html break;
    }
    if (!-f $request_filename) {
       proxy_pass http://mongrel;
       break;
    }
}
... }

}

Everything is perfect and as expected, except handling of errors that
come from mongrel. There are a few referring sites that overload the
error.log of Nginx with errors like these:

proxy: Error reading from remote server returned by /some/path referer:
http://somesite.com

(70014)End of file found: proxy: error reading status line from remote
server localhost, referer:

For a period of 30min. the error.log reaches ~500MB of those errors.

If I access a mongrel_instance directly with the same problematic
request I get this from the instance output (not logs):

Sun Aug 17 12:14:43 +0300 2008: HTTP parse error, malformed request
(127.0.0.1): #<Mongrel::HttpParserError: Invalid HTTP format, parsing
fails.>

It’s obvious where the problem is with mongrel, however it overloads the
whole server with Nginx error logs and CPU load.

I got similar problems when using Apache as proxy_balancer, however the
rate of logs is far slower, so it doesn’t overload the server.

Any help or idea will be appreciated. I’m sure that our application
deployment isn’t so unique, so someone else should have had the same
problem.