Upstream server temporarily disabled while reading response header from upstream

Server NGINX
1

We are runnging nginx reverse proxy on windows, the upstream consists of
two
lunixes based Apache/PHP Backend Servers. There is only one PHP
application
to be proxied. After starting nginx everything works fine, but then the
backends become partly unresponsive and nginx is logging

2016/05/24 18:42:27 [warn] 5116#4660: *8863 upstream server temporarily
disabled while reading response header from upstream, client:
77.23.138.88,
server: wahl2.hannover-stadt.de, request: “GET / HTTP/1.1”, upstream:
“http://192.168.57.14:80/wrs/”, host: “wahl2.hannover-stadt.de”
2016/05/24 18:42:27 [error] 5116#4660: *8863 upstream timed out (10060:
A
connection attempt failed because the connected party did not properly
respond after a period of time, or established connection failed because
connected host has failed to respond) while reading response header from
upstream, client: 77.23.138.88, server: wahl2.hannover-stadt.de,
request:
“GET / HTTP/1.1”, upstream: “http://192.168.57.14:80/wrs/”, host:
“wahl2.hannover-stadt.de”
2016/05/24 18:42:36 [warn] 5116#4660: *8862 upstream server temporarily
disabled while reading response header from upstream, client:
77.23.138.88,
server: wahl2.hannover-stadt.de, request: “GET / HTTP/1.1”, upstream:
“http://192.168.57.14:80/wrs/”, host: “wahl2.hannover-stadt.de”
2016/05/24 18:42:36 [error] 5116#4660: *8862 upstream timed out (10060:
A
connection attempt failed because the connected party did not properly
respond after a period of time, or established connection failed because
connected host has failed to respond) while reading response header from
upstream, client: 77.23.138.88, server: wahl2.hannover-stadt.de,
request:
“GET / HTTP/1.1”, upstream: “http://192.168.57.14:80/wrs/”, host:
“wahl2.hannover-stadt.de”
2016/05/24 18:42:37 [warn] 3560#5948: *8870 upstream server temporarily
disabled while reading response header from upstream, client:
77.23.138.88,
server: wahl2.hannover-stadt.de, request: “GET
/modules/setflashsession.php?set=0 HTTP/1.1”, upstream:
“http://192.168.57.14:80/wrs/modules/setflashsession.php?set=0”, host:
“wahl2.hannover-stadt.de”, referrer: “http://wahl2.hannover-stadt.de/”
2016/05/24 18:42:37 [error] 3560#5948: *8870 upstream timed out (10060:
A
connection attempt failed because the connected party did not properly
respond after a period of time, or established connection failed because
connected host has failed to respond) while reading response header from
upstream, client: 77.23.138.88, server: wahl2.hannover-stadt.de,
request:
“GET /modules/setflashsession.php?set=0 HTTP/1.1”, upstream:
“http://192.168.57.14:80/wrs/modules/setflashsession.php?set=0”, host:
“wahl2.hannover-stadt.de”, referrer: “http://wahl2.hannover-stadt.de/”
2016/05/24 18:42:47 [warn] 3560#5948: *8870 upstream server temporarily
disabled while reading response header from upstream, client:
77.23.138.88,
server: wahl2.hannover-stadt.de, request: “GET
/modules/setflashsession.php?set=0 HTTP/1.1”, upstream:
“http://192.168.57.88:80/wrs/modules/setflashsession.php?set=0”, host:
“wahl2.hannover-stadt.de”, referrer: “http://wahl2.hannover-stadt.de/”
2016/05/24 18:42:47 [error] 3560#5948: *8870 upstream timed out (10060:
A
connection attempt failed because the connected party did not properly
respond after a period of time, or established connection failed because
connected host has failed to respond) while reading response header from
upstream, client: 77.23.138.88, server: wahl2.hannover-stadt.de,
request:
“GET /modules/setflashsession.php?set=0 HTTP/1.1”, upstream:
“http://192.168.57.88:80/wrs/modules/setflashsession.php?set=0”, host:
“wahl2.hannover-stadt.de”, referrer: “http://wahl2.hannover-stadt.de/”

Googeling for " upstream server temporarily disabled while reading
response
header from upstream" brings up a few russian links but no clue whats
the
reason for this. There is no load on the system, both backend servers
and
nginx are idle.

Posted at Nginx Forum:

2

Hello!

On Tue, May 24, 2016 at 01:15:54PM -0400, hheiko wrote:

connection attempt failed because the connected party did not properly
respond after a period of time, or established connection failed because
connected host has failed to respond) while reading response header from
upstream, client: 77.23.138.88, server: wahl2.hannover-stadt.de, request:
“GET / HTTP/1.1”, upstream: “http://192.168.57.14:80/wrs/”, host:
“wahl2.hannover-stadt.de”

[…]

Googeling for " upstream server temporarily disabled while reading response
header from upstream" brings up a few russian links but no clue whats the
reason for this. There is no load on the system, both backend servers and
nginx are idle.

The “upstream server temporarily disabled” warning means exactly
this: the server was disabled due to failures, and will not be
considered for balancing for some time.

This warning is logged in addition to normal logging of errors
(“upstream timed out” in your case), and means that number of
errors observed by nginx crossed the max_fails threshold.

More details on max_fails / fail_timeout can be found here:

http://nginx.org/en/docs/http/ngx_http_upstream_module.html#max_fails

–
Maxim D.
http://nginx.org/