Proxy_pass to backend (varnish): delivered ip?

Hey,
maybe the solution is really simple, but i can’t find it.

nginx (1.2.1) handles ssl and proxies the traffic to the backend
(varnish,
which also handles http):

location / {
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_pass http://127.0.0.1:80;
}

The requests are sent to varnish, in the varnish logs i see them, they
look
like (shortened):

127.0.0.1 - - [30/Jan/2013:16:06:54 +0100] …

So varnish receives 127.0.0.1 from nginx, but i want varnish to receive
the
external ip.

For testing reasons i switched the ip to the lan ip (which varnish also
listens to):

location / {
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_pass http://192.168.0.1:80;
}

And now the requests in varnish log look like:

192.168.0.1 - - [30/Jan/2013:16:16:25 +0100] …

Well, where’s the error? How can it be done that varnish receives the
external ip (smth. like 84.156.23.145) from nginx? I suppose it’s quite
simple but i can’t see it… :-/

Posted at Nginx Forum:

On Wed, Jan 30, 2013 at 10:33:25AM -0500, revirii wrote:

Hi there,

The requests are sent to varnish, in the varnish logs i see them, they look
like (shortened):

127.0.0.1 - - [30/Jan/2013:16:06:54 +0100] …

So varnish receives 127.0.0.1 from nginx, but i want varnish to receive the
external ip.

The connection to varnish comes from the address 127.0.0.1. That’s what
it logs here.

And now the requests in varnish log look like:

192.168.0.1 - - [30/Jan/2013:16:16:25 +0100] …

The connection to varnish comes from the address 192.168.0.1. That’s
what it logs here.

Well, where’s the error? How can it be done that varnish receives the
external ip (smth. like 84.156.23.145) from nginx? I suppose it’s quite
simple but i can’t see it… :-/

varnish could see the “true” remote_addr in the X-Real-IP: http header
that you send, if it looked there.

If you want varnish to log the contents of that header, or to log that
instead of what it sees as the connecting ip address, you’ll be better
off to ask in a place where people know varnish.

f

Francis D. [email protected]

Hey,

thx for the fast answer :slight_smile:

The connection to varnish comes from the address 127.0.0.1. That’s
what it logs here.

The connection to varnish comes from the address 192.168.0.1. That’s
what it logs here.

But why? The only difference is the proxy_pass statement:

proxy_pass http://127.0.0.1:80;
vs.
proxy_pass http://192.168.0.1:80;

No other changes were done, and no changes in varnish config.

varnish could see the “true” remote_addr in the X-Real-IP: http header
that you send, if it looked there.

If you want varnish to log the contents of that header, or to log that
instead of what it sees as the connecting ip address, you’ll be better
off to ask in a place where people know varnish.

Hm, would be interesting which param varnish checks. It can’t be
$remote_addr, so it has to be the address nginx proxies to (127.0.01 or
192.168.0.1). Very strange. So it seems to be a varnish problem? :-/

revirii

Posted at Nginx Forum:

Hello,

Am 30.01.2013 17:13, schrieb revirii:

varnish could see the “true” remote_addr in the X-Real-IP: http
header
that you send, if it looked there.

Hm, would be interesting which param varnish checks. It can’t be
$remote_addr, so it has to be the address nginx proxies to (127.0.01
or
192.168.0.1). Very strange. So it seems to be a varnish problem? :-/

You can see the right X-Forwarded-For header with tcpdump.

Take a look at the log format varnish uses. I don’t know how to
configure varnish, but with Apache you have to change your LogFormat
from

LogFormat “%h %l %u %t “%r” %>s %b “%{Referer}i”
“%{User-Agent}i”” combined

to something like

LogFormat “%{X-Forwarded-For}i %l %u %t “%r” %>s %b
“%{Referer}i””%{User-Agent}i"" nginx

rgds, Axel


Never argue with an idiot; people watching may not tell the difference

On Wed, Jan 30, 2013 at 11:13:38AM -0500, revirii wrote:

Hi there,

proxy_pass http://192.168.0.1:80;

No other changes were done, and no changes in varnish config.

Look at the routing table on your nginx server.

If it connects to 127.0.0.1, it will connect from 127.0.0.1 (which
is one of the nginx server’s addresses).

If it connects to 192.168.0.1, it will connect from 192.168.0.1
(which is one of the nginx server’s addresses).

(Probably, if it connects to 192.168.0.2 (which is on a different
machine), it will connect from 192.168.0.1.)

Hm, would be interesting which param varnish checks. It can’t be
$remote_addr, so it has to be the address nginx proxies to (127.0.01 or
192.168.0.1).

No, it is the address that the connection to varnish comes from.
Because
of your specific setup, that happens to match the address that nginx
connects to. But try connecting to varnish from some other machine and
you’ll see the difference.

Very strange. So it seems to be a varnish problem? :-/

It’s usually considered a feature that the source address of a
connection
is logged. There is nothing nginx can do to hide its source address.

What you want is something non-standard. Possibly there’s a varnish
configuration to allow it.

f

Francis D. [email protected]

Good Morning,

LogFormat “%{X-Forwarded-For}i %l %u %t "%r" %>s %b
"%{Referer}i""%{User-Agent}i"” nginx

Good idea. I’ll check if varnish is able to do that.

thx
revirii

Posted at Nginx Forum:

On Thu, Jan 31, 2013 at 02:13:39AM -0500, revirii wrote:

Hi there,

If it connects to 127.0.0.1, it will connect from 127.0.0.1 (which
is one of the nginx server’s addresses).

If it connects to 192.168.0.1, it will connect from 192.168.0.1
(which is one of the nginx server’s addresses).

Hm, the said nginx vhost doesn’t use 127.0.0.1 or a lan address in
combination with port 80 - varnish listens on *.80. the nginx ssl vhost
listens on an “real” ip and passes the requets to varnish on 127.0.0.1:80.
Or is there a misunderstanding on my side?

Yes.

nginx as a server listens on whatever ip:port pairs are in the
configuration.

nginx as a client uses the operating system’s facilities to set the
client connection information.

When nginx talks to varnish, nginx is the client and varnish is the
server.

Ok, so what your’re trying to tell me is:
if i proxy_pass to varnish with “proxy_pass http://127.0.0.1:80;” nginx uses
some localhost address (although it can’t be 127.0.0.1:80, which is used by
varnish) to connect to varnish, and varnish sees this localhost address

The proxy_pass connection comes from the address and port that nginx
chooses – which is usually “whatever the OS chooses”, because nginx
doesn’t care to choose. In this case, the address will be 127.0.0.1,
and the port will be something unpredictable you usually don’t care
about.

if i proxy_pass to varnish with “proxy_pass http://real_ip:80;” nginx uses
the real ip (although it can’t be real_ip:80, which is used by varnish) to
connect to varnish, and varnish sees this real_ip address

If real_ip is configured on the machine, then yes. If not, then the
OS will choose the appropriate locally-configured address (plus an
unpredictable port) to make the connection.

If so, there’s nothing i can do within nginx config, as it’s simply not
possible. And varnish config or log is the (only) place where i can achieve
this.

That part is correct.

f

Francis D. [email protected]

Good Morning,

If it connects to 127.0.0.1, it will connect from 127.0.0.1 (which
is one of the nginx server’s addresses).

If it connects to 192.168.0.1, it will connect from 192.168.0.1
(which is one of the nginx server’s addresses).

Hm, the said nginx vhost doesn’t use 127.0.0.1 or a lan address in
combination with port 80 - varnish listens on *.80. the nginx ssl vhost
listens on an “real” ip and passes the requets to varnish on
127.0.0.1:80.
Or is there a misunderstanding on my side?

No, it is the address that the connection to varnish comes from.
Because
of your specific setup, that happens to match the address that nginx
connects to. But try connecting to varnish from some other machine and
you’ll see the difference.

Well, it’s not that special, i think.
http: request → varnish on real ip → nginx backend on 127.0.0.1:81
https: request → nginx on real ip:443 → proxy_pass to varnish on
127.0.01.80 → nginx backend on 127.0.0.1:81
(If necessary i could draw a small picture)

It’s usually considered a feature that the source address of a
connection
is logged. There is nothing nginx can do to hide its source address.

What you want is something non-standard. Possibly there’s a varnish
configuration to allow it.

Ok, so what your’re trying to tell me is:
if i proxy_pass to varnish with “proxy_pass http://127.0.0.1:80;” nginx
uses
some localhost address (although it can’t be 127.0.0.1:80, which is used
by
varnish) to connect to varnish, and varnish sees this localhost address

if i proxy_pass to varnish with “proxy_pass http://192.168.0.1:80;”
nginx
uses some lan address (although it can’t be 192.168.0.1:80, which is
used by
varnish) to connect to varnish, and varnish sees this lan address

if i proxy_pass to varnish with “proxy_pass http://real_ip:80;” nginx
uses
the real ip (although it can’t be real_ip:80, which is used by varnish)
to
connect to varnish, and varnish sees this real_ip address

If so, there’s nothing i can do within nginx config, as it’s simply not
possible. And varnish config or log is the (only) place where i can
achieve
this.

thx a lot
revirii

Posted at Nginx Forum: