Change client_body_buffer_size from 16K to 256K made the nginx logs size from 50M to 1G

Server NGINX
1

Hi Team,

I always use below configuration to record the post date of my webserver
(for security resaon)

http {

…

log_format main '$remote_addr - $remote_user [$time_local]
“$request” ’
'$status $body_bytes_sent “$http_referer” ’
‘“$http_user_agent” “$http_x_forwarded_for”’;
log_format plog '$remote_addr - $remote_user [$time_local]
“$request” ’
'$status $body_bytes_sent “$http_referer” ’
‘“$http_user_agent” “$http_x_forwarded_for” “$request_body”’;

server {
…

  location ~ \.php$ {
  try_files $uri =404;
  if ($request_method = POST){
    return                 484;
    break;
  }
  error_page               484 = @post;
  fastcgi_index  index.php;
  fastcgi_param  SCRIPT_FILENAME  /scripts$fastcgi_script_name;
  include        fastcgi_params;
  fastcgi_pass   backend;
}
location @post{
  internal;
  access_log /web/log/post.log  plog;
  try_files $uri =404;
  fastcgi_index  index.php;
  fastcgi_param  SCRIPT_FILENAME  /scripts$fastcgi_script_name;
  include        fastcgi_params;
  fastcgi_pass   backend;
  }

}
}

And today I found that the size the post.log was almost 1G everyday, in
the
past it’s only 50M. The only thing I changed recently is the buffer_size

From :

client_header_buffer_size 64k;
large_client_header_buffers 4 32k;
client_body_buffer_size 16k;
client_max_body_size 50m;
fastcgi_buffer_size 64k;
fastcgi_buffers 4 64k;
fastcgi_busy_buffers_size 128k;
fastcgi_temp_file_write_size 128k;

To:
client_header_buffer_size 4k;
large_client_header_buffers 4 32k;
client_body_buffer_size 256k;
client_max_body_size 8m;
fastcgi_buffer_size 128k;
fastcgi_buffers 256 16k;
fastcgi_busy_buffers_size 256k;
fastcgi_temp_file_write_size 256k;

Is that because I changed client_body_buffer_size from 16K to 256K
caused
the size change of the log file?
For now the client_body_buffer_size is big enough, when users upload a
file,
then nginx will put it into the buffer instead of a temp file, and then
also
write this file into post.log? Am I right?

If I’m right, then how can I exclude file upload from the post log? The
$request_uri for the upload is mod=swfupload.

Can anyone help?

Thanks

Posted at Nginx Forum:

2

On Wed, May 11, 2016 at 01:39:14AM -0400, meteor8488 wrote:

log_format plog '$remote_addr - $remote_user [$time_local] “$request” ’
'$status $body_bytes_sent “$http_referer” ’
‘“$http_user_agent” “$http_x_forwarded_for” “$request_body”’;

What do you think that the last element of that log_format definition
does?

http://nginx.org/r/$request_body

Is that because I changed client_body_buffer_size from 16K to 256K caused
the size change of the log file?
For now the client_body_buffer_size is big enough, when users upload a file,
then nginx will put it into the buffer instead of a temp file, and then also
write this file into post.log? Am I right?

Yes.

If I’m right, then how can I exclude file upload from the post log? The
$request_uri for the upload is mod=swfupload.

If you don’t want the request body logged, don’t log the request body.

If you don’t want the request body logged for one $request_uri only,
you can finish handling that in a specific location{} and use a
different
access_log there.

f

Francis D. [email protected]

3

Thanks for your quickly response.

One more question,

for client_body_buffer_size 16K, if the $request_body >16K, it seems
nginx
will put the request body into a temp file, and then no logs in log
file,
even though I enabled the request log. Does that mean the best way to
keep
the post log is to enable client_body_in_file_only? But the thing is
enable
client_body_in_file_only will slow down nginx.

So is there any better way to achieve that?

Posted at Nginx Forum:

4

Hi all, I just updated my configuration files as following

location ~ .php$ {
try_files $uri =404;
if ($arg_mod = “upload” ) {
return 485;
break;
}
if ($request_method = POST){
return 484;
break;
}
error_page 484 = @post;
error_page 485 = @flash;
fastcgi_param SCRIPT_FILENAME /scripts$fastcgi_script_name;
include fastcgi_params;
fastcgi_pass backend;
}
location @post{
internal;
access_log /web/log/post.log plog;
try_files $uri =404;
fastcgi_param SCRIPT_FILENAME /scripts$fastcgi_script_name;
include fastcgi_params;
fastcgi_pass backend;
}
location @flash{
internal;
access_log /web/log/flash.log main;
try_files $uri =404;
fastcgi_param SCRIPT_FILENAME /scripts$fastcgi_script_name;
include fastcgi_params;
fastcgi_pass backend;
}

I’m using if to check whether user want to upload a file or not.

But I know that if is evil, so how can achieve the same result without
using
if?

Posted at Nginx Forum:

5

On Wednesday 11 May 2016 09:19:58 meteor8488 wrote:

    break;
  try_files $uri =404;
  fastcgi_pass   backend;
  }

I’m using if to check whether user want to upload a file or not.

But I know that if is evil, so how can achieve the same result without using
if?

[…]

Please, check the “if=” parameter of the “access_log” directive.
See the docs:
http://nginx.org/en/docs/http/ngx_http_log_module.html#access_log

wbr, Valentin V. Bartenev