We got lots of “No route to host” in Nginx error log.
’
2010/01/05 22:28:01 [error] 29366#0: *26063920 connect() failed (113: No
route to host) while connecting to upstream, client: 60.217.152.205,
server: test.com, request: GET /herundong/index.php HTTP/1.1, upstream: http://x.y.z.a:80/herundong/index.php, host: events.test.com ’
Using curl on the Nginx proxy server like
curl -v -H “Host: events.test.com” http://x.y.z.a/herundong/index.php
works as expected.
I searched in Nginx source code, and find even no ‘No route to host’ in
its source code.
Nginx version: 0.7.63
Backend are Apache servers.
Current Nginx connections ~2k;
Current Apache Connections ~800(counting all active and inactive
connections)
I saw on some connections of apache, there was a high Send-Q on
established nginx-apache connections.
Current Apache load average is under 1;
We got lots of “No route to host” in Nginx error log.
’
2010/01/05 22:28:01 [error] 29366#0: *26063920 connect() failed (113: No route to host) while connecting to upstream, client: 60.217.152.205, server: test.com, request: GET /herundong/index.php HTTP/1.1, upstream: http://x.y.z.a:80/herundong/index.php, host: events.test.com
’
“No route to host” is the error string of system call connect. It’s
error value is 113. It means the server can’t find the route to
the backend server. This’s the network problem, not nginx.
I find that error to , My Configuration is 4 upstreams, three upstreams
is 1 cpuã€2G memory,one is 2cpuã€4Gmemory,the 2cpu’s doesn’t have this
error,but the three 1cpus have this error,those server all in the same
LAN ,anyone have the suggestion?
On Tue, 2010-03-23 at 08:26 -0400, zhangyun112112 wrote:
I find that error to , My Configuration is 4 upstreams, three
upstreams is 1 cpuã€2G memory,one is 2cpuã€4Gmemory,the 2cpu’s doesn’t
have this error,but the three 1cpus have this error,those server all
in the same LAN ,anyone have the suggestion?
As å´”çŽ‰æ¾ said, this is a network issue, not a Nginx issue. You should
check the proper function and configuration of both your LAN and the
machines attached to it.
“No route to host” is not nginx error, if you use linux, please man
route
for detail information
please make sure that the firewall does not block the port to your
backend
We have that problem for awhile at our site. During the last two days,
I investigated it and I pretty sure that it is cause by the netfilter
firewall. Our backend servers are running RHEL5 and the 'iptables, rule
that accept new connections from the proxy looks like:
-A RH-Firewall-1-INPUT -m state --state NEW -m tcp -p tcp -s
iii.jjj.kkk.lll --dport 9005 -j ACCEPT
iii.jjj.kkk.lll is the IP address of the proxy server.
And the last rule of the chain ‘RH-Firewall-1-INPUT’ is:
-A RH-Firewall-1-INPUT -j REJECT --reject-with icmp-host-prohibited
Yesterday I inserted a new rule in between in order to log incoming
packets from the proxy that are going to be rejected. The rule looks
like:
-A RH-Firewall-1-INPUT -s iii.jjj.kkk.lll -j LOG
The result was that for each ‘No route to host’ error logged in the
error_log file on the proxy server, there is a corresponding rejected
packet logged on one of the backend servers. That means that our
‘iptables’ rules need a little bit of tuning.
I thought of two solutions:
Remove the ‘state match’ test in the ACCEPT rule so the new rule
looks like:
Add a new rule to ‘iptables’ which issues a REJECT with options
‘tcp-reset’ for the problematics packets. I hope that the proxy will
reissue the request while receiving the RST flag from the backend. The
rule should be inserted just before the last one and it should look
like:
Tonight I will try the solution number 2. If the proxy react as I think
it will, That will be a better solution than the number 1 which I will
try eventually if the solution number 2 won’t work.
-p tcp -s iii.jjj.kkk.lll --dport 9005 -j ACCEPT
Yesterday I inserted a new rule in between in
tuning.
REJECT with options ‘tcp-reset’ for the
better solution than the number 1 which I will try
eventually if the solution number 2 won’t work.
Hope that this message will help you.
Sorry for the poor english.
Paul
Hi again,
Finally the solution #2 solves the problem. I just made a last minute
modification ( I restrict the rule to the -SYN packet) to the rule in
‘iptables’. The right rule looks like this: