Lua possibilities/limitations

[dubstep]

Hi Guys,
I need a little help with the lua module. My ultimate aim is to import
variables into nginx from outside source.
Here is my code, you can probably tell what I’m trying to do:

location /proxy {
proxy_pass http://127.0.0.1:8012/;
}

location /main {
set $myvar ‘’;
set $limit_rate 99;
content_by_lua ’
local res = ngx.location.capture(“/proxy”,{ args = { bla = “moo” } })
if res.status == ngx.HTTP_OK then
for k,v in pairs(res.header) do
if k == “X-Limit-Rate”
then
ngx.var.myvar
= v
end
end
return
else
ngx.exit(res.status)
end
';

set $limit_rate $myvar;
add_header X-Test $myvar;
add_header X-Limit $limit_rate;

}
}

The output is quite confusing:

curl -I http://localhost:101/main
HTTP/1.1 200 OK
Server: nginx/0.8.54
Date: Sun, 13 Feb 2011 03:29:00 GMT
Content-Type: application/octet-stream
Connection: keep-alive
X-Test: 100
X-Limit: 99

Am I going about this in the right way? As I say, my ultimate aim is to
set a whole bunch of nginx vars from outside…

Many thanks
Richard

[rkoskela]

On Sun, Feb 13, 2011 at 12:02 PM, Richard K.
[email protected] wrote:

Hi Guys,

I need a little help with the lua module. My ultimate aim is to import
variables into nginx from outside source.

Your code below is totally wrong because you do not understand nginx
request phases well enough

1. Your content_by_lua always runs after any write-phase directrives
   like “set” and “if” even if you put “set $limit_rate $myvar;” after it
   in the config file.

2. Your content_by_lua for the /main location does not generate any
   response bodies (using ngx.print or ngx.say in Lua). So you see the
   empty response body on the client side.

BTW, your “add_header” directive runs in the output filter phase, so
your X-Test header gets the value set by your Lua code, which is
expected. But your X-Limit gets the old value because “set $limit_rate
$myvar” runs before $myvar gets the new value in Lua and $limit_rate
is still the old value. (For why “set” runs after your content_by_lua
code, see above).

FWIW, rewrite_by_lua and access_by_lua also run after those “set”,
“if”, “rewrite” directives provided by ngx_rewrite.

Am I going about this in the right way? As I say, my ultimate aim is to set
a whole bunch of nginx vars from outside…

For this purpose, rewrite_by_lua or access_by_lua will almost be
sufficient. But if you want to calculate variable values using
response headers (from ngx_proxy or ngx_fastcgi), then you need to
issue subrequests in content_by_lua before ngx_lua gets the
“output_header_filter_by_lua” directive implemented

Cheers,
-agentzh

[rkoskela]

On Mon, Feb 14, 2011 at 12:08 PM, agentzh [email protected] wrote:

Your code below is totally wrong because you do not understand nginx
request phases well enough

Just want to mention that plain nginx.conf programming can be evil
because IMHO 99.99% of the nginx users on this planet do not and I bet
will not grok all of these details about nginx phases and various
pitfalls regarding “if”, “add_header”, and etc. Mixing these with or
without Lua can be confusing. And that’s why we’re advocating pure Lua
scripting wherever feasible. Add your headers and doing your code
branches using pure Lua because it’s a mature and well designed
language (as compared to the nginx.conf language).

Some mature nginx users or developers might disagree here. But that’s
our opinions and recommendations anyway

Cheers,
-agentzh

[rkoskela]

Hi,

…
On 13/02/2011 06:02, Richard K. wrote:

location /main {

set $myvar ‘’;

set $limit_rate 99;

…

set $limit_rate $myvar;

As well as the points agentzh made, there is also the case here that
$limit_rate does a check on the values that it is set to, and it must be
a valid size (internally this is done by having a set_handler attached
to the variable - something most variables don’t have).

Here, you are trying to set it to ‘’ (i.e. empty), and this is invalid,
and it will :
(a) throw an error in the error log, saying that it’s an ‘invalid size’
(b) do nothing to the value, therefore leaving it at 99 rather than
setting it to ‘’

With the config you have, most variables would result in your headers
being
…

X-Test: 100

X-Limit:

But you see X-Limit: 99 because of this reason.

Cheers,

Marcus.

[rkoskela]

Thanks for the help guys, I’m getting somewhere now, config file feels
so script like it’s an easy trap to fall in to
Please consider this example…

location /main {
set $myvar ‘’;
set $limit_rate 99;
content_by_lua ’
local res = ngx.location.capture(“/proxy”,{ args = { bla = “moo” } })
if res.status == ngx.HTTP_OK then
for k,v in pairs(res.header) do
if k == “X-Limit-Rate”
then
ngx.var.myvar
= v
ngx.var.limit_rate
= v
end
end
return
else
ngx.exit(res.status)
end
';

add_header X-Test $myvar;
add_header X-Limit $limit_rate;

}

curl -I http://localhost:101/main
HTTP/1.1 200 OK
Server: nginx/0.8.54
Date: Mon, 14 Feb 2011 23:40:06 GMT
Content-Type: application/octet-stream
Connection: keep-alive
X-Test: 100
X-Limit: 99

Is there something special with $limit_rate stopping me to set it? Is it
a string/int issue?

Regards,
Richard K.

From: Eugaia [mailto:[email protected]]
Sent: 14 February 2011 10:35
To: [email protected]
Subject: Re: lua possibilities/limitations

Hi,

…
On 13/02/2011 06:02, Richard K. wrote:
location /main {
set $myvar ‘’;
set $limit_rate 99;
…

set $limit_rate $myvar;
As well as the points agentzh made, there is also the case here that
$limit_rate does a check on the values that it is set to, and it must be
a valid size (internally this is done by having a set_handler attached
to the variable - something most variables don’t have).

Here, you are trying to set it to ‘’ (i.e. empty), and this is invalid,
and it will :
(a) throw an error in the error log, saying that it’s an ‘invalid size’
(b) do nothing to the value, therefore leaving it at 99 rather than
setting it to ‘’

With the config you have, most variables would result in your headers
being
…
X-Test: 100
X-Limit:

But you see X-Limit: 99 because of this reason.

Cheers,

Marcus.

[rkoskela]

On Tue, Feb 15, 2011 at 7:46 AM, Richard K.
[email protected] wrote:

Is there something special with $limit_rate stopping me to set it? Is it a
string/int issue?

Yeah, $limit_rate is a special variable defined in the http core module:

http://wiki.nginx.org/NginxHttpCoreModule#.24limit_rate

For now, ngx_lua’s “ngx.var.foo = xxx” assignment does not support
such special variables. This will change in a future release.

I’ve confirmed for you that replacing $limit_rate with $limit gives
the expected headers Not sure if you really want to set $limit_rate
to control transfer limit rate though.

Cheers,
-agentzh

[rkoskela]

On Tue, Feb 15, 2011 at 4:57 PM, Eugaia [email protected] wrote:

This explains a lot. By this do you mean it doesn’t support variables
with set handlers, or also changeable variables with different get handlers?

The former only, I think

Cheers,
-agentzh

[rkoskela]

On Tue, Feb 15, 2011 at 12:27 PM, agentzh [email protected] wrote:

For now, ngx_lua’s “ngx.var.foo = xxx” assignment does not support
such special variables. This will change in a future release.

I’ve just committed a fix for nginx variable assignment in Lua to
ngx_lua’s git master HEAD. Now all the nginx variables with a
set_handler is properly handled (like $args and $limit_rate).
Furthermore, only changeable nginx variables will be honored, so
writing to $arg_PARAM will result in a Lua exception (ready to be
caught by pcall/xpcall if LuaJIT 2.0 is used).

Here’s the modified version of your sample config that should work for
you:

location /proxy {
    proxy_pass http://127.0.0.1:8012/;
}

location /main {
    set $limit_rate 99;
    rewrite_by_lua '
        local res = ngx.location.capture("/proxy",{ args = { bla =

“moo” } })
if res.status == ngx.HTTP_OK then
for k,v in pairs(res.header) do
if k == “X-Limit-Rate” then
ngx.var.limit_rate = v
end
end

            return
        end

        ngx.exit(res.status)
';

add_header X-Limit $limit_rate;

}

My modifications mainly correct the following mistake in your previous
sample:

• content_by_lua always runs in the content phase while ngx_rewrite’s
  “set” command always runs at the earlier rewrite phase. So your
  content_by_lua will always run after your “set” commands even if you
  put it before them.

As a bonus, I also add reading support for nginx regex group capturing
variables like $1, $2, $3, and etc, in Lua.

Happy Lua hacking!

Best,
-agentzh

[rkoskela]

Hi,

On 15/02/2011 06:27, agentzh wrote:

For now, ngx_lua’s “ngx.var.foo = xxx” assignment does not support
such special variables. This will change in a future release.
This explains a lot. By this do you mean it doesn’t support
variables with set handlers, or also changeable variables with different
get handlers?

Marcus.

[rkoskela]

Thanks!
I have tested and this part works