Static files bad loading time

Server NGINX
1

Hey guys,

I have a dedicated server at iWeb.com (i3-540 + 8GB RAM). It has free
bandwidth, average load is around 0.05-0.2 during the day and I/O wait
ratio
is very low now.

However, sometimes my Nginx 1.4.2 loads in a browser 1MB image for like
10-30 seconds. Sometimes it takes 2 seconds like it should. My bandwidth
is
also free, of course. I made some tests like MTR, ping, traceroute –
everything is OK.

Updated Nginx to 1.8.0 but nothing’s changed.

Here is my Nginx config:

worker_processes 4;
worker_rlimit_nofile 12400;

events {
worker_connections 32768;
}

http {
include mime.types;
default_type application/octet-stream;

log_format  main  '$remote_addr - $remote_user [$time_local]

$request ’
'“$status” $body_bytes_sent “$http_referer” ’
‘“$http_user_agent” “$http_x_forwarded_for”’;

sendfile        on;
tcp_nopush     on;
tcp_nodelay    on;

gzip  on;
gzip_min_length    1400;
gzip_proxied    any;
gzip_types    text/plain text/xml application/xml

application/x-javascript text/javascript text/css text/json;
gzip_comp_level 6;

map $http_host $root_dir {
    hostnames;
}
root $root_dir;

server {
    listen       188.88.88.88:80;
    server_name  domain.com www.domain.com;
    access_log  /nginx/logs/domain.com-access.log  main;

    location / {
        proxy_pass        http://domain.com:8080/;
        proxy_redirect    off;
        proxy_set_header   Host             $host;
        proxy_set_header   X-Real-IP        $remote_addr;
        proxy_set_header   X-Forwarded-For

$proxy_add_x_forwarded_for;

        client_max_body_size       20m;
        client_body_buffer_size    128k;

        proxy_connect_timeout      90;
        proxy_send_timeout         90;
        proxy_read_timeout         90;

        proxy_buffer_size          128k;
        proxy_buffers              4 256k;
        proxy_busy_buffers_size    256k;
        proxy_temp_file_write_size 256k;

        root /home/www/domain.com;
    }
}

}

I use server for a couple of websites (PHP+MySQL). One of websites is an
image hosting. Usually it serves images less than 1Mb but I created a
couple
of tools so user can upload a bigger image now. But still 3-5Mb max.

Does anybody know what causes this loading lag?

Thanks in advance!

Posted at Nginx Forum:

2

I use CentOS 6.6.

Posted at Nginx Forum:

3

On Sat, Apr 25, 2015 at 02:01:33PM -0400, grigory wrote:

Hi there,

However, sometimes my Nginx 1.4.2 loads in a browser 1MB image for like
10-30 seconds. Sometimes it takes 2 seconds like it should.

Does anybody know what causes this loading lag?

The configuration you show doesn’t seem to tell nginx to serve any
static files.

Perhaps the port-8080 server can tell you more about what is happening?

f

Francis D. [email protected]

4

So, Francis… Do you have any idea on my problem?

Posted at Nginx Forum:

5

Sorry, I forgot to add the following part of the config (from server’s
block):

    # Static files location
    location ~*

^.+.(jpg|jpeg|gif|png|ico|css|zip|tgz|gz|rar|bz2|doc|xls|exe|pdf|ppt|txt|tar|wav|bmp|rtf|js)$
{
if ($args ~* “^download”) {
add_header Content-Disposition “attachment;
filename=$1”;
}

        expires    30d;
        root   /home/www/domain.com;
    }

Posted at Nginx Forum:

6

On Sun, Apr 26, 2015 at 06:11:32AM -0400, grigory wrote:

Hi there,

    # Static files location
    location ~*

^.+.(jpg|jpeg|gif|png|ico|css|zip|tgz|gz|rar|bz2|doc|xls|exe|pdf|ppt|txt|tar|wav|bmp|rtf|js)$

So - you have your configuration; you make a request; sometimes you get
the response quickly, but sometimes you get the response slowly.

Can you tell from nginx logs whether the slowness is due to
slow-read-from-disk, or slow-write-to-client, or something else?

Can you find any pattern in the requests which respond more slowly than
you want? Certain browsers, certain times of day, anything like that?

If you make the request from the machine itself, so network issues
should
be minor, does it still show sometimes being slow?

Good luck with it,

f

Francis D. [email protected]

7

Hi Francis,

Can you tell from nginx logs whether the slowness is due to
slow-read-from-disk, or slow-write-to-client, or something else?

Could you please tell me how to check this out?
My nginx logs do not contain this sort of information.

Can you find any pattern in the requests which respond more slowly than
you want? Certain browsers, certain times of day, anything like that?

Unfortunately, I didn’t find any pattern. It’s just sometimes loads in 2
seconds and in another time – in 10-15 seconds. I mean same 300KB image
within a couple of refreshes in a browser. I’ve tested the problem on
different browsers and different times of day – no luck.

If you make the request from the machine itself, so network issues should
be minor, does it still show sometimes being slow?

When I make request from machine itself, the image loads pretty fast.

Posted at Nginx Forum:

8

On Thursday 07 May 2015 23:27:44 shahzaib shahzaib wrote:

Also, default keepalive_timeout value is 65sec due to which your current

The keepalive_timeout has nothing to do with the maximum number of
concurrent connections per second.

wbr, Valentin V. Bartenev

9

Hi,

There are some tweaks required to nginx configurations. If the same
image which usually takes second to response can takes upto 10-20
seconds
to load, the wide guess would be exceeding concurrent connections at
peak
traffic. The directive worker_rlimit_nofile value is set much lower as
compare to worker_connections. Nginx uses upto 2 file descriptors per
connections, so i would suggest to increase worker_rlimit_nofile value
to
124000.

Also, default keepalive_timeout value is 65sec due to which your current
nginx configuration is not optimized to serve more than 2000 concurrent
connections. Here’s how :

(Worker_process)4 * 32768(worker_connections) / 65(Keepalive_timeout ==
2016 connections per seconds.

So i would suggest to decrease keepalive_timeout to 5sec directive and
increase worker_connections to 60000.

Also make sure to decrease timeout values.

Regards.
Shahzaib

10

Well, reducing keepalive_timeout and increasing the values of
worker_connections resolved our issue. Following is the reference we
used
to tweak nginx config :

Thanks.
Shahzaib

On Fri, May 8, 2015 at 4:42 PM, Valentin V. Bartenev [email protected]

11

On Friday 08 May 2015 18:05:51 shahzaib shahzaib wrote:

Well, reducing keepalive_timeout and increasing the values of
worker_connections resolved our issue. Following is the reference we used
to tweak nginx config :

Optimizing Nginx for High Traffic Loads

This reference is quite inaccurate. Don’t trust arbitrary articles in
the internet.

See here for the detailed explanation:
http://mailman.nginx.org/pipermail/nginx/2015-May/047460.html

wbr, Valentin V. Bartenev

12

On Thu, May 07, 2015 at 01:56:02PM -0400, grigory wrote:

Hi there,

Can you tell from nginx logs whether the slowness is due to
slow-read-from-disk, or slow-write-to-client, or something else?

Could you please tell me how to check this out?
My nginx logs do not contain this sort of information.

I’m actually not sure how to go about that. Possibly there will be
details in the debug log? But you do not want to run the debug log on
a busy system that only sporadically shows the problem.

Possibly a lighter way of trying to identify a pattern is to include
$request_time in you normal access_log.

Then when you see the slowness, you can identify the request in the
logs,
and see is there any pattern that way – does it always and only happen
when there are more than 100 other concurrent requests; or at that
start of a minute when something else on the system is busy starting;
or something that is common to these requests and not to others.

(Or maybe the common feature is not something that nginx can see.)

If you make the request from the machine itself, so network issues should
be minor, does it still show sometimes being slow?

When I make request from machine itself, the image loads pretty fast.

If that remains true when you make the request a hundred times, that
suggests that the problem is outside of nginx’s control, in the system
networking or (more likely) the network outside the nginx server.

Good luck with it,

f

Francis D. [email protected]

13

Right, thanks.

Btw, we used another nginx official doc for optimization and the most
effective optimization parameter was tweaking the backlog from default
512
to 4096 in nginx listen directive.

Regards.
Shahzaib

On Fri, May 8, 2015 at 6:18 PM, Valentin V. Bartenev [email protected]