Caching issues with nginx as reverse-proxy

Server NGINX
1

Hi community,

currently I am serving files with a size about 1,5G (static without
dynamic content) using a hand full of nodes and nginx in reverse proxy
setup.
Caching works, …but not as expected. During the requests nginx creates
a lot of temp caching files that grow up to the size of the origin file
delivered from backend server. It’s kinda weird that even if the
inactive caching time out is not reached, the file is randomly
downloaded again from backend.

I guess that I just simply got blind by the configure that affects that.

/etc/nginx/nginx.conf

proxy_cache_path /var/tmp/nginx-cache/rproxy levels=1:2
keys_zone=rproxy:260m max_size=5g inactive=260m use_temp_path=off;
proxy_cache_key “$scheme$request_method$host$request_uri$is_args$args”;
proxy_cache_valid 200 302 45m;
proxy_cache_valid 404 1m;

/etc/nginx/sites-available/domain.tld

server {
listen 80;# default_server;
listen [::]:80;# default_server;

 server_name domain.tld;

 location / {
 return 301 https://$host$request_uri;
 }

}

 server {

 listen 443 ssl http2; #default_server;
 listen [::]:443 ssl http2; #default_server;

 ssl    on;
 ssl_certificate    /etc/letsencrypt/live/domain.tld/fullchain.pem;
 ssl_certificate_key

/etc/letsencrypt/live/domain.tld/privkey.pem;

 server_name domain.tld;

 location / {

proxy_buffering on;

proxy_cache_revalidate on;
proxy_cache_lock on;
proxy_cache rproxy;
proxy_cache_use_stale error timeout http_500 http_502 http_503
http_504;
proxy_pass https://rproxy;
}

}

Nginx 1.11.1 is used. And the frontends got only 1gb ram and 10 gb of
disk space.

I am looking forward for any advice that could bring me some steps
forward to the right configuration :slight_smile:

Thanks in advance!

Best regards,
Daniel

2

Hello!

On Thu, Jun 30, 2016 at 12:02:29AM +0200, [email protected] wrote:

currently I am serving files with a size about 1,5G (static without dynamic
content) using a hand full of nodes and nginx in reverse proxy setup.
Caching works, …but not as expected. During the requests nginx creates a
lot of temp caching files that grow up to the size of the origin file
delivered from backend server. It’s kinda weird that even if the inactive
caching time out is not reached, the file is randomly downloaded again from
backend.

Multiple temporary files can be created when the response is not
yet cached and there are multiple requests to the resource in
question. Once the response is cached it will be returned from
cache to subsequent requests, and no additional temporary files
will appear.

To mininize parallel caching by multiple simultaneous requests
there is the “proxy_cache_lock” directive. Though for 1.5G files
default timeouts it uses may not be enough, consider tuning
proxy_cache_lock_age and proxy_cache_lock_timeout:

http://nginx.org/r/proxy_cache_lock_age
http://nginx.org/r/proxy_cache_lock_timeout

Additionally, consider using the slice module. It was specially
designed to improve caching of large files, see here:

http://nginx.org/en/docs/http/ngx_http_slice_module.html

[…]

proxy_cache_path /var/tmp/nginx-cache/rproxy levels=1:2
keys_zone=rproxy:260m max_size=5g inactive=260m use_temp_path=off;

Note well: with “max_size” set to 5g the cache in question can
hold about 3 1.5G files. It may not be what you want for
effective caching.

[…]

–
Maxim D.
http://nginx.org/