Limiting parallel requests by source IP address

Server NGINX
1

I am using nginx as a reverse proxy to a Ruby on Rails application using
the
unicorn server on multiple load-balanced application servers. This
configuration allows many HTTP requests to be serviced in parallel. I’ll
call the total number of parallel requests that can be serviced ‘P’,
which
is the same as the number of unicorn processes running on the
application
servers.

I have many users accessing the nginx server and I want to ensure that
no
single user can consume too much (or all) of the resources. There are
existing plugins for this type of thing: limit_conn and limit_req. The
problem is that it looks like these plugins are based upon the request
rate
(i.e. requests per second). This is a less than ideal way to limit
resources
because the rate at which requests are made does not equate to the
amount of
load the user is putting on the system. For example, if the requests
being
made are simple (and quick to service) then it might be OK for a user to
make 20 per second. However, if the requests are complex and take a
longer
time to service then we may not want a user to be able to make more than
1
of these expensive requests per second. So it is impossible to choose a
rate
that allows many quick requests, but few slow ones.

Instead of limiting by rate, it would be better to limit the number of
parallel requests a user can make. So if the total system can service
P
parallel requests we would limit any one user to say P/10 requests. So
from
the perspective of any one user our system appears to have 1/10th of the
capacity that it really does. We don’t need to limit the capacity to
P/number_of_users because in practice most users are inactive at any
point
in time. We just need to ensure that no matter how many requests, fast
or
slow, that one user floods the system with, they can’t consume all of
the
resources and so impact other users.

Note that I don’t want to return a 503 error message to a user who tries
to
make more than P/10 requests at once. I just want to queue the next
request
so that it will eventually execute, just more slowly.

I can’t find any existing plugin for Nginx that does this. Am I missing
something?

I am planning to write a plugin that will allow me to implement resource
limits in this way. But I am curious if anyone can see a hole in this
logic,
or an alternative way to achieve the same thing.

Thanks,

Chris.

Posted at Nginx Forum:

2

First, you need to define ‘user’, which is not a trivial problem.
Unless you use the commercial subscription, it is hard to tie a
connection
to a session. You can use components in fornt of nginx to identify them
(usually with cookies).
Thus ‘user’ in nginx FOSS usually means ‘IP address’.

Now, you have the limit_conn
http://nginx.org/en/docs/http/ngx_http_limit_conn_module.html module,
as
you noticed, which allows you to limit (as its name suggests)
connections,
and not rate (which is the job of… the limit_req
http://nginx.org/en/docs/http/ngx_http_limit_req_module.html module).
You need to set a key on the zone to the previously told value in order
to
identify a ‘user’. Then the limit_conn directive allows you to set the
connections limit.

To make the limit number vary, you will need to automatically
(periodically) re-generate some included configuration file and reload
nginx configuration, as it is evaluated once before requests are
processed.
You might also use some lua scripting to change it on the fly, I
suppose.

However, you won’t be able to do anything else by rejecting extra
connections with a 503 (or another customisable HTTP code). The client
will
then need to try to connect again following a certain shared protocol
after
some conditions are met (time?).

===

Another idea (looks like dirty, but you will decide on that):
If you know what time a specific request needs to complete, you can try
to
play with limit_req to limit connections, using the burst property of
this
directive to stack waiting (but not rejected) connections.
Say you want to limit connections to 5, but accept up to 10 of them, all
that in 30s, you can use a limit of 5 associated with a rate of 2/m and
a
burst of 5.

All that requires testing, provided I correctly understood your
specifications.
My 2 cents,

B. R.

3

Hi,

For my purposes IP address is an acceptable definition of a user. In any
case I would use the commercial subscription if it would help with this
problem.

Rate limiting doesn’t help because I don’t know ahead of time whether a
user
will make many fast requests, or fewer slow requests - and in fact a
user
might make a mix of fast and slow requests, so rate limiting wouldn’t be
effective at limiting the resources being used.

Regards,

Chris.

B.R. Wrote:

module, as
To make the limit number vary, you will need to automatically
after
Say you want to limit connections to 5, but accept up to 10 of them,

which

problem is that it looks like these plugins are based upon the
make 20 per second. However, if the requests are complex and take a
parallel requests a user can make. So if the total system can
slow, that one user floods the system with, they can’t consume all
missing

nginx mailing list
[email protected]
nginx Info Page

Posted at Nginx Forum: