server {
…
location @proxy {
include /etc/nginx/firewall.default;
proxy_pass http://127.0.0.1:8080;
…
}
location ~ ^.+.php$ {
content_by_lua ‘ngx.exec(“@proxy”);’;
}
location / {
try_files $uri $uri/ @proxy;
}
}
Basically, everything that cannot be found by nginx, as well as php
requests, are sent to the proxy
Now, note the filter.default file in the @proxy location. I use this
to run some tests on these requests for security and my logs show them
catching all sorts of exploit attempts.
Anyway, when I have the following (simplified) in firewall.default …
if ($http_user_agent ~* libwww ) {
return 403;
}
… everything is fine. When a php request is made, libwww user agents
are denied and others get the php output.
When I use the following (simplified) rewrite_by_lua equivalent instead
…
rewrite_by_lua ’
if ngx.var.http_user_agent == “libwww” then
ngx.exit(ngx.HTTP_FORBIDDEN)
end
';
The php file is downloaded. Obviously I don’t have the “libwww” when
testing so I suppose the lua “if” block is skipped at which point the
physical php file is found and sent to the user as is and the
proxy_pass directive is not run.
Looks similar to the sort of unexpected behaviour from the rewrite
module’s “if”.
Any ideas what gives? Why isn’t rewrite_by_lua behaving like the rewrite
module?
When I use the following (simplified) rewrite_by_lua equivalent instead …
rewrite_by_lua ’
if ngx.var.http_user_agent == “libwww” then
ngx.exit(ngx.HTTP_FORBIDDEN)
end
';
Note that you’re using “==” here in Lua which is exact string comparison
The php file is downloaded. Obviously I don’t have the “libwww” when
testing so I suppose the lua “if” block is skipped at which point the
physical php file is found and sent to the user as is and the
proxy_pass directive is not run.
Which version of ngx_lua are you using? Please show me your “nginx -V”
output? And which OS are you using? I’ve tested your example with
ngx_lua git master HEAD on Slackware Linux x86_64 and do not have any
issues
Also, enabling --with-debug in your nginx build and show me the
relevant sections of your error.log on the debug error log level will
be helpful too
Note that you’re using “==” here in Lua which is exact string comparison
It is a simplified config. I actually run lua’s string.find first and
test for a hit.
I know ngx.re.match with the “i” modifier would be better but it does
not work … maybe I need to update lua module version. I thought I’ll
look into that later.
Anyway, the exact implementation will cause the “if” block to be
skipped.
The php file is downloaded. Obviously I don’t have the “libwww” when
testing so I suppose the lua “if” block is skipped at which point the
physical php file is found and sent to the user as is and the
proxy_pass directive is not run.
Which version of ngx_lua are you using? Please show me your “nginx -V”
output? And which OS are you using? I’ve tested your example with
ngx_lua git master HEAD on Slackware Linux x86_64 and do not have any
issues
It is a simplified config. I actually run lua’s string.find first and
test for a hit.
I know ngx.re.match with the “i” modifier would be better but it does
not work
The ngx.re API was first introduced in the v0.2.1rc11 release. There’s
no wonder that it does not work for you because you’re using v0.2.0
… maybe I need to update lua module version. I thought I’ll
look into that later.
Surely you need. Please do that before other attempts
Which version of ngx_lua are you using? Please show me your “nginx -V”
output? And which OS are you using? I’ve tested your example with
ngx_lua git master HEAD on Slackware Linux x86_64 and do not have any
issues
Centos 5.7 i386.
lua module is 0.2.0
Sigh, your ngx_lua module is way too old
There’s a bunch of important fixes regarding ngx.exec() and ngx.exit()
since the ngx_lua v0.2.0 release.
The latest version is v0.3.1rc9 and please try it out.
Also, enabling --with-debug in your nginx build and show me the
relevant sections of your error.log on the debug error log level will
be helpful too
I’ll get this later.
Please upgrade your ngx_lua module to the latest version before doing
this
There’s a bunch of important fixes regarding ngx.exec() and ngx.exit()
since the ngx_lua v0.2.0 release.
The latest version is v0.3.1rc9 and please try it out.
OK thanks. I need to do a build for Nginx 1.0.8 and update my 3rd
party modules with new nginx builds.
I don’t install the rc versions of 3rd party modules though. I only
use full releases which means 0.3.0 in this case … expect if 0.3.1
is released in the interim of course.
Anyway, nice to know ngx_lua is not “evil” (as long as it is 0.2.1+!)
I had thought the php files were being downloaded as text files but
this is not strictly true. Files are downloaded whenever php is
called but they are always empty with a “0” status code.
The associated access log output for the request in the debug log in
the paste bin is: xx.xxx.xx.xx - - [12/Oct/2011:18:20:50 +0000] “GET
/help/ HTTP/1.1” 0 0 “http://testsite/index.php” “Mozilla/5.0
(Macintosh; Intel Mac OS X 10_5_8) AppleWebKit/534.50.2 (KHTML, like
Gecko) Version/5.0.6 Safari/533.22.3”
if method == nil then
method = ngx.re.match(ngx.var.request_method, "HEAD", "i")
if method == nil then
ngx.exit(ngx.HTTP_BAD_REQUEST)
end
end
end
';
BTW, you could have written the Lua code like this:
local m = ngx.re.match(ngx.var.request_method, “GET|POST|HEAD”, “io”)
if not m then
ngx.exit(400)
end
The ngx.re.match uses the same PCRE engine as ngx_rewrite’s “if”
anyway. Also, because the regex argument is a constant here, the “o”
flag can be specified to cache the compiled regex object on the global
level.
The debug log data you’ve provided contains a lot of references to the
location @pretty_urls which is missing from your original nginx.conf
snippet. Could you please provide a minimized but complete nginx.conf
that can reproduce this problem as well as the corresponding debug.log
snippet?
I had thought the php files were being downloaded as text files but
this is not strictly true. Files are downloaded whenever php is
called but they are always empty with a “0” status code.
This information is invaluable I’ve already reproduced this bug of
ngx_lua on my side and fixed it in my local tree.
Basically, calling ngx.exec to jump to a named location is buggy in
that ngx_http_named_location() in the nginx core does not clear module
ctx structures as ngx_http_internal_redirect() so ngx_lua should have
cleared its ctx itself.
I’ll tag another rc release of ngx_lua for you to try when I finish
running the whole test suite
Thank you very much for the bug report! Hopefully it’ll be no longer
evil
if ($request_method !~* (GET|HEAD|POST) ) {
return 400;
}
… while this sends php files as text downloads …
rewrite_by_lua ’
method = ngx.re.match(ngx.var.request_method, “GET”, “i”)
if method == nil then
method = ngx.re.match(ngx.var.request_method, “POST”, “i”)
if method == nil then
method = ngx.re.match(ngx.var.request_method, “HEAD”, “i”)
if method == nil then
ngx.exit(ngx.HTTP_BAD_REQUEST)
end
end
end
';
The debug log data you’ve provided contains a lot of references to the
location @pretty_urls which is missing from your original nginx.conf
snippet. Could you please provide a minimized but complete nginx.conf
that can reproduce this problem as well as the corresponding debug.log
snippet?
server {
…
location @pretty_urls {
# some rewrite rules to php
}
location @proxy {
include /etc/nginx/firewall.default;
proxy_pass http://127.0.0.1:8080;
…
}
location ~ ^.+.php$ {
content_by_lua ‘ngx.exec(“@proxy”);’;
}
location / {
try_files $uri $uri/ @pretty_urls;
}
}
If request ends in “.php” it is sent to @proxy by php location. If not
“.php”, Nginx tries to find and serve resource. If not found, sent to @pretty_urls where it will either be rewritten to “.php” or a 404
error returned.
Basically, calling ngx.exec to jump to a named location is buggy in
that ngx_http_named_location() in the nginx core does not clear module
ctx structures as ngx_http_internal_redirect() so ngx_lua should have
cleared its ctx itself.
I see. Don’t know much about this but does this mean the httpRewrite
module does clear this ctx? I.E. why doesn’t this affect the standard
rewrite module?
I’ll tag another rc release of ngx_lua for you to try when I finish
running the whole test suite
Thank you very much for the bug report! Hopefully it’ll be no longer evil
Thanks are to you.
# some rewrite rules to php
try_files $uri $uri/ @pretty_urls;
}
}
If request ends in “.php” it is sent to @proxy by php location. If not
“.php”, Nginx tries to find and serve resource. If not found, sent to @pretty_urls where it will either be rewritten to “.php” or a 404
error returned.
PS: I had earlier thought the issue might be with the content_by_lua
‘ngx.exec(“@proxy”);’; line but when I tried to use the following …
Basically, calling ngx.exec to jump to a named location is buggy in
that ngx_http_named_location() in the nginx core does not clear module
ctx structures as ngx_http_internal_redirect() so ngx_lua should have
cleared its ctx itself.
I see. Don’t know much about this but does this mean the httpRewrite
module does clear this ctx? I.E. why doesn’t this affect the standard
rewrite module?
For ngx_lua, for example, it does have a flag field named
“entered_content_phase” in its context object (ctx) which marks
whether the current request has entered the content phase. So when
doing an internal redirection to a named location, this field must be
cleared out or some bad things will happen in rewrite_by_lua because
ngx_lua now believes it’s in a content_by_lua not rewrite_by_lua.
Well, there’re more flags like this in ngx_lua ctx. For ngx_rewrite,
it’s probably safe just because of its simplicity.
Well, I still believe it is a bug in ngx_http_named_location function
in the nginx core. The thumb of rule is to avoid using named locations
like @proxy but use normal locations configured with the “internal”
directive. And we’ve been keeping doing this in our production apps.
Rebuilt with 0.31rc11. Also noticed note about, and implemented,
module loading order.
Everything seems straight as an arrow and no evil going ons detected so
far.
This did not work due to exactly the same issue with ngx.exec + named
locations. Please try the latest v0.3.1rc10 release mentioned earlier.
It should also work now
Unfortunately, I am still seeing the “0 0” outputs in my logs on (only
some) requests. I had noticed that some pages just seemed to have the
“loading” wheel spin for ever and on looking, I found the instances.
It is not affecting standard php requests anymore from what I can see
but it seems some Ajax type requests where php is being used for
dynamic js (! need to look closer into how I set those up) and with
phpMyAdmin which is aliased somewhere.
Probably has to do with named locations as you mentioned earlier but I
prefer to use these so I have rolled back to my back to my 1.0.6
install so as to get back my ngx_rewrite based setup which works fine
with this.
