Fastcgi chunked data

Server NGINX
1

I have a fastcgi server connection that send chunks of data. I would
like to have the chucks of data written to downstream, not buffered.

I want to send chunked data from the server, but nginx buffers the data,
I want to flush the data immediately.

I have found the “trunk” 1.1.0 code almost works.

1> The event happens
2> the pipe is read
3> something is checked???
4> the data is buffered, not written downstream
5> try to write down stream, but no buffer, Why?
5> checks upstream again
6> starts timer again

I have traced the code but I get lost at

[debug] 2842#0: *4 pipe recv chain: -2
src/event/ngx_event_pipe.c:272
n = p->upstream->recv_chain(p->upstream, chain);

        ngx_log_debug1(NGX_LOG_DEBUG_EVENT, p->log, 0,
                       "pipe recv chain: %z", n);

AND

why there is no buffer to write?
[debug] 2842#0: *4 pipe write downstream: 1 0000000000000000

2011/08/18 17:26:01 [debug] 2842#0: *4 post event 0000000001FF1BF0
2011/08/18 17:26:01 [debug] 2842#0: *4 post event 0000000002005400
2011/08/18 17:26:01 [debug] 2842#0: *4 delete posted event
0000000002005400
2011/08/18 17:26:01 [debug] 2842#0: *4 http upstream request:
“/mafw/weiqi.psh?” 1
2011/08/18 17:26:01 [debug] 2842#0: *4 http upstream dummy handler
2011/08/18 17:26:01 [debug] 2842#0: *4 delete posted event
0000000001FF1BF0
2011/08/18 17:26:01 [debug] 2842#0: *4 http upstream request:
“/mafw/weiqi.psh?” 0
2011/08/18 17:26:01 [debug] 2842#0: *4 http upstream process upstream
2011/08/18 17:26:01 [debug] 2842#0: *4 pipe read upstream: 1
2011/08/18 17:26:01 [debug] 2842#0: *4 readv: 2:4096
2011/08/18 17:26:01 [debug] 2842#0: *4 pipe recv chain: 144
2011/08/18 17:26:01 [debug] 2842#0: *4 readv: 2:4096
2011/08/18 17:26:01 [debug] 2842#0: *4 readv() not ready (11: Resource
temporarily unavailable)
2011/08/18 17:26:01 [debug] 2842#0: *4 pipe recv chain: -2
2011/08/18 17:26:01 [debug] 2842#0: *4 pipe buf free s:0 t:1 f:0
0000000001FC9750, pos 0000000001FC9750, size: 3368 file: 0, size: 0
2011/08/18 17:26:01 [debug] 2842#0: *4 pipe buf free s:0 t:1 f:0
0000000001FC7730, pos 0000000001FC7730, size: 0 file: 0, size: 0
2011/08/18 17:26:01 [debug] 2842#0: *4 pipe write downstream: 1
0000000000000000
2011/08/18 17:26:01 [debug] 2842#0: *4 pipe write busy: 0
2011/08/18 17:26:01 [debug] 2842#0: *4 pipe write: out:0000000000000000,
f:0
2011/08/18 17:26:01 [debug] 2842#0: *4 pipe read upstream: 0
2011/08/18 17:26:01 [debug] 2842#0: *4 pipe buf free s:0 t:1 f:0
0000000001FC9750, pos 0000000001FC9750, size: 3368 file: 0, size: 0
2011/08/18 17:26:01 [debug] 2842#0: *4 pipe buf free s:0 t:1 f:0
0000000001FC7730, pos 0000000001FC7730, size: 0 file: 0, size: 0
2011/08/18 17:26:01 [debug] 2842#0: *4 event timer del: 20:
1313659596126
2011/08/18 17:26:01 [debug] 2842#0: *4 event timer add: 20:
60000:1313659621126

2

That’s a lot of debugging when you could just have read the wiki
instead: Module ngx_http_fastcgi_module

“If you want to disable buffering to disk for all replies that are
greater than the FastCGI buffers and transfer data synchronously to the
client set fastcgi_max_temp_file_size to 0.”

Posted at Nginx Forum:

3

Hello!

On Fri, Aug 19, 2011 at 08:39:02AM +0800, Shaun savage wrote:

I have a fastcgi server connection that send chunks of data. I would
like to have the chucks of data written to downstream, not buffered.

I want to send chunked data from the server, but nginx buffers the data,
I want to flush the data immediately.

As of now nginx will always buffer fastcgi response up to full buffer
(see fastcgi_buffers directive) filled.

If you really need data to be flushed immediately, you may try the
patch from here:

http://mailman.nginx.org/pipermail/nginx/2011-August/028324.html

It is to address completely different problem, but as a side
effect with the patch complete fastcgi records will be sent without
further buffering.

Maxim D.

4

I am not reading a file from disk, I am generating the data from a
program.
I added

fastcgi_max_temp_file_size 0;

but is still buffers the data in the FastCGI buffers.

the size i am chunking is only 144 bytes