Using Nginx with chunking

Server NGINX
1

Hello,

I have nginx configured to send files in chunks to remote clients. The
clients will contact the server to send it in chunks of 1 MB each. I am
using nginx version 1.8.0

[root@ph-rdu-external-download-01 sbin]# ./nginx -V

nginx version: nginx/1.8.0

built by gcc 4.4.7 20120313 (Red Hat 4.4.7-4) (GCC)

built with OpenSSL 1.0.1e-fips 11 Feb 2013

TLS SNI support enabled

configure arguments: --user=nginx --group=nginx
–prefix=/usr/local/nginx --lock-path=/var/lock/subsys/nginx
–with-ld-opt=-Wl,-rpath,/usr/lib64 --with-debug
–add-module=/root/integ/ngx_devel_kit-master/
–add-module=/root/integ/lua-nginx-module-0.9.15
–error-log-path=/var/log/nginx/error.log
–http-log-path=/var/log/nginx/access.log
–without-http_autoindex_module --without-http_fastcgi_module
–with-http_ssl_module --without-http_geo_module
–without-http_empty_gif_module --without-http_ssi_module
–without-http_userid_module --without-mail_pop3_module
–without-mail_imap_module --without-mail_smtp_module
–without-http_uwsgi_module --without-http_scgi_module
–with-http_realip_module --with-http_gzip_static_module
–with-http_stub_status_module --with-ipv6

Currently I am running into an issue, where if the last chunk size is
very small (few bytes), the server doesnt seem to the send the chunk
correctly because of which the clients would re-try again for the same
chunk. For some reason it is not able to send the last chunk. I have
attached the debug logs to the thread. If you see the logs the last
three writes seem to be for the same chunk of size 6672 bytes

However if the last chunk size relatively large, then download succeeds
without any issues. The issue is seen when the last chunk size is very
small.

My current nginx configuration is as follows

http {

    # Logging format

    log_format main '$remote_addr - $remote_user [$time_local]  '

'"$request_length" “$request_time” ’

'"$request" $status $bytes_sent ' '"$body_bytes_sent" "$bytes_sent"

';

    default_type                    application/octet-stream;

    include                         mime.types;

    keepalive_timeout               300 300;

    keepalive_requests              8000;

    charset                         utf-8;

    source_charset                  utf-8;

    # Check if it makes errors.

    ignore_invalid_headers      off;

    recursive_error_pages           on;

    sendfile                        on;

    server_tokens                   off;

    tcp_nodelay                     on;

    tcp_nopush                      off;

}

Regards,
Arun

2

Hello!

On Wed, Aug 12, 2015 at 09:30:15AM +0000, Arun John (arujohn) wrote:

Hello,

I have nginx configured to send files in chunks to remote
clients. The clients will contact the server to send it in
chunks of 1 MB each. I am using nginx version 1.8.0

You confuse chunks and range requests. These are distinct things
in HTTP.

Currently I am running into an issue, where if the last chunk
size is very small (few bytes), the server doesn’t seem to the
send the chunk correctly because of which the clients would
re-try again for the same chunk. For some reason it is not able
to send the last chunk. I have attached the debug logs to the
thread. If you see the logs the last three writes seem to be for
the same chunk of size 6672 bytes

What makes you think that the problem is with nginx, and not with
the client?

–
Maxim D.
http://nginx.org/

3

Hi Maxim,

Sorry for the confusion.

But even for range requests, it should return the bytes requested,
correct? Am I missing any configuration?

Regards,
Arun

On 8/12/15, 6:23 PM, “nginx on behalf of Maxim D.”

4

If you see the logs the last three writes seem to be for the same chunk of
size 6672 bytes

I imagine this is a typo (and not what the client expects) because the
length is 6272 bytes.

But even for range requests, it should return the bytes requested,
correct? Am I missing any configuration?

Well you can always try to test it with something like curl to see if
the
problem is not really on the particular client:

curl --header “Range: bytes=16777216-16783487” http://yourfile
or
curl -r 16777216-16783487 http://yourfile

There are some SSL errors in the debuglog though.

rr

5

Hello!

On Wed, Aug 12, 2015 at 01:05:13PM +0000, Arun John (arujohn) wrote:

But even for range requests, it should return the bytes requested,
correct? Am I missing any configuration?

The question is the same: what makes you think that nginx doesn’t
return bytes requested? As per logs you’ve provided, everything
is correctly returned to the client.

–
Maxim D.
http://nginx.org/