Transfer-encoding chunked hex markers appearing in output?

Server NGINX
1

I’ve written a very simple script which acts as a specific proxy for XML
feeds, reading from a source URL and passing the content straight
through. It’s trivial and it’s working fine when run as a standalone
HTTP server (via node.js).

However when I put nginx in front of it I get what appear to be the
chunk hex markers in the output - which obviously renders the XML
invalid. Here is an example of the output, note the “f32” at line 1, and
the repeated “1000” on lines 53 & 85, the
“58a” on lines 125 & 137 and so on.

Here is my (reduced) nginx conf:

server {
listen 80;
server_name feeder;
location / {
proxy_pass http://127.0.0.1:8888;
}
}

I’ve confirmed that the chunks themselves don’t contain the spurious
data (by dumping them to a file and checking with dhex), and apart from
the expected Server: change the HTTP headers are the same with and
without nginx:

HTTP/1.1 200
Server: nginx/1.0.12
Date: Mon, 05 Mar 2012 02:02:38 GMT
Content-Type: text/xml; charset=UTF-8
Connection: keep-alive
etag: rM1jGkNUA9AzVj2iMQN9KWwLgcQ
last-modified: Mon, 05 Mar 2012 02:07:53 GMT
expires: Mon, 05 Mar 2012 02:08:03 GMT
cache-control: private, max-age=0
x-content-type-options: nosniff
x-xss-protection: 1; mode=block
transfer-encoding: chunked

I’m at a loss how to proceed and I’m hoping someone on the list
recognises the problem.

In case it’s relevant, here is the code for the script, with the lines
that do the proxying highlighted:

2

I should add that this is with nginx 1.0.12 and node 0.6.11.

3

On Mon, 2012-03-05 at 11:22 +0800, Henry wrote:

However when I put nginx in front of it I get what appear to be the
chunk hex markers in the output - which obviously renders the XML
invalid. Here is an example of the output, note the “f32” at line 1, and
the repeated “1000” on lines 53 & 85, the
“58a” on lines 125 & 137 and so on.

There is an experimental patch that allows Nginx to speak HTTP 1.1 to
backends (including chunked responses):

http://mailman.nginx.org/pipermail/nginx/2011-August/028324.html

I’m afraid I don’t know if it will apply against 1.0.12.

Regards,
Cliff

4

Hello!

On Mon, Mar 05, 2012 at 11:22:52AM +0800, Henry wrote:

}
Connection: keep-alive
etag: rM1jGkNUA9AzVj2iMQN9KWwLgcQ
last-modified: Mon, 05 Mar 2012 02:07:53 GMT
expires: Mon, 05 Mar 2012 02:08:03 GMT
cache-control: private, max-age=0
x-content-type-options: nosniff
x-xss-protection: 1; mode=block
transfer-encoding: chunked

I’m at a loss how to proceed and I’m hoping someone on the list
recognises the problem.

According to RFC2616, “A server MUST NOT send transfer-codings
to an HTTP/1.0 client.” That is, correct solution is to fix your
script - it must not return chunked data to nginx (as nginx uses
HTTP/1.0 in requests to backends).

Alternatively, you may upgrade to 1.1.x which has support for
HTTP/1.1 to backends and is able to recognize and remove chunked
encoding in such situations.

Maxim D.

5

On 05/03/12 at 12:34pm, Maxim D. wrote:

According to RFC2616, “A server MUST NOT send transfer-codings
to an HTTP/1.0 client.” That is, correct solution is to fix your
script - it must not return chunked data to nginx (as nginx uses
HTTP/1.0 in requests to backends).

Alternatively, you may upgrade to 1.1.x which has support for
HTTP/1.1 to backends and is able to recognize and remove chunked
encoding in such situations.

Hi Maxim - thanks for clarifying this for me. I’ve tried the previously
mentioned patch without success, I’ll see if I can upgrade to 1.1.x but
there are a lot of other sites on this server so that may not be wise
just yet.

Thanks again!

Henry.

6

On 04/03/12 at 09:48pm, Cliff W. wrote:

There is an experimental patch that allows Nginx to speak HTTP 1.1 to
backends (including chunked responses):

upstream keepalive - call for testing

I’m afraid I don’t know if it will apply against 1.0.12.

Thanks Cliff - I’ll see if I can get it to work with the patch.