Limit_req is not working with dynamically extracted user address

Server NGINX
1

Hi, if our HTTP block looks like below where we find IP from
X-Forwarded-For
using perl module it looks like zone and limit_req are not using correct
variable $user_real_ip or it is reset right after logging.

The nginx_access_log shows the first logged field correctly as
$user_real_ip

  • which is first element of comma separated IP addresses in
    X-Forwarded-For
    header from different hopes. User request comes thorough several
    different
    CDN and DDOS services.

But when limit_req kicks in it would take for some reason last element
in
same header X-Forwarded-For which of course not our intention.

Can you help understand why and what we are doing wrong?

Also to note we are using 1.6.2 version. We tried to upgrade it to 1.8
version but it didn’t help.

Thanks,

Sergey

Example of nginx_access_log and following nginx_error_log

555.182.61.171 - - - www.my.com - [17/Mar/2016:17:44:15 -0400] “GET
/our/api/here HTTP/1.1” 503 270 0.000 “-” “Java/1.8.0_51”
“555.182.61.171,
333.101.98.188” -

2016/03/17 17:44:15 [error] 19382#0: *8 limiting requests, excess: 5.613
by
zone “one”, client: 333.101.98.188, server: www.my.com, request: “GET
/our/api/here HTTP/1.1”, host: “www.my.com”

HTTP block

http {
include /etc/nginx/mime.types;
include /etc/nginx/proxy.conf;
index index.html index.htm index.php;

…
perl_set $user_real_ip ’

    sub {
        my $r = shift;
        my $str = $r->header_in("X-Forwarded-For");
        my @fields = split /,/, $str;
        my $real_ip = $fields[0];
        return $real_ip;
    }

';

log_format  main  '$user_real_ip - $remote_addr - $remote_user -

$host -
[$time_local] “$request” ’
'$status $body_bytes_sent $request_time
“$http_referer” ’
‘“$http_user_agent” “$http_x_forwarded_for”
$http_cf_ray’;

sendfile        on;

keepalive_timeout  20;

limit_req_zone $user_real_ip zone=one:50m rate=1r/s;

include /etc/nginx/conf.d/*.conf;

}

www.conf location

2

Hello!

On Fri, Mar 18, 2016 at 09:45:58AM -0400, malish8632 wrote:

same header X-Forwarded-For which of course not our intention.

Can you help understand why and what we are doing wrong?

How did you found that limit_req uses a wrong element?

Note:

2016/03/17 17:44:15 [error] 19382#0: *8 limiting requests, excess: 5.613 by
zone “one”, client: 333.101.98.188, server: www.my.com, request: “GET
/our/api/here HTTP/1.1”, host: “www.my.com”

The “client: 333.101.98.188” is the client address as automatically
logged by nginx for all error messages (also known as $remote_addr).
It’s not related to the limit string used by limit_req.

[…]

';

Note well: this can be easily tricked by clients using a fake
address in initial X-Forwarded-For header. You may want to use
the realip module instead, it allows to trust only known proxies,
see here:

http://nginx.org/en/docs/http/ngx_http_realip_module.html

–
Maxim D.
http://nginx.org/

3

Hi Maxim,
thank you for quick response.

How did you found that limit_req uses a wrong element?

We don’t know if this is limit_req - in reality we were just looking
into
logs and I guess that’s what confused us. We observed those IPs and
rolled
back the changes as we assumed that all requests from CDN or DDOS
Service
were blocked.

The only way to I guess to verify that our current schema works is to
use
some arbitrary IP and see if our requests are blocked rather then CDN
service IP is blocked.

We’ve looked into
http://nginx.org/en/docs/http/ngx_http_realip_module.html
and not sure if it is going to work.

As you saw one of the examples we have other services in front of us.
There are 2 cases:
User → DDOS Service → Our NGINX - X-Forwarded-For ex:
555.182.61.171, 333.101.98.188
User → CDN → DDOS Service → Our NGINX - X-Forwarded-For ex:
555.182.61.171, 444.1.3.56, 555.12.34.567, 333.101.98.188

Will realip module able to identify real IP of end user?
Should we set CIDR of both DDOS Service and CDN Service as real ip
tables:

set_real_ip_from 192.168.1.0/24;
set_real_ip_from 192.168.2.1;
set_real_ip_from 2001:0db8::/32;

Thanks again.

Sergey

Posted at Nginx Forum:

4

Hello!

On Fri, Mar 18, 2016 at 10:48:56AM -0400, malish8632 wrote:

How did you found that limit_req uses a wrong element?

We don’t know if this is limit_req - in reality we were just looking into
logs and I guess that’s what confused us. We observed those IPs and rolled
back the changes as we assumed that all requests from CDN or DDOS Service
were blocked.

The only way to I guess to verify that our current schema works is to use
some arbitrary IP and see if our requests are blocked rather then CDN
service IP is blocked.

Ok, so no problem here.

Will realip module able to identify real IP of end user?
Should we set CIDR of both DDOS Service and CDN Service as real ip tables:

set_real_ip_from 192.168.1.0/24;
set_real_ip_from 192.168.2.1;
set_real_ip_from 2001:0db8::/32;

The realip module uses last non-trusted address from the header
(assuming real_ip_recursive is set). So you have to instruct it
to trust addresses of your DDoS mitigation service and CDN, e.g.:

set_real_ip_from ;
set_real_ip_from ;
real_ip_header X-Forwarded-For;
real_ip_recursive on;

–
Maxim D.
http://nginx.org/