Nginx trying to connect to upstream host which is down

Server NGINX
1

Hi,

Note: I had to change the hostnames and domain names.

nginx version: nginx/1.4.6 (Ubuntu)

I have the following the configuration:
upstream ssl-app-cluster {
ip_hash;
server app01.example.com:8443 max_fails=1 fail_timeout=60s;
server app02.example.com:8443 max_fails=1 fail_timeout=60s;
}

And during the Application deployment(rolling deployment) to those
backend
servers(app0{1,2}.example.com) the port 8443 will not be available for
60sec. So, during the deployment I tried hitting the
server(api.example.com)
continuously to know, how nginx is routing the traffic, when one of the
servers in down. As I am using ip_hash, my ip is bound to
app01.example.com
initially.

Deployment process:

  1. I am running the curl command to hit the server api.example.com in a
    for
    loop and I am being served by app01(I see this on error log with debug
    enabled).
  2. Deployment process has taken down the app02 host for upgrading the
    application on it. This doesn’t effect anything, as my IP is bound to
    app01
    and I am being served by app01.
  3. Now, the deployment has taken down app01 for deployment. So, I see
    there’s a “111 connection refused error” and nginx tried to connect to
    “http
    next upstream” which is app02. which is successful.
  4. But, nginx has to wait for 60sec(fail_timeout) to establish
    connection to
    app01, But I see nginx is trying to connect to app01 immediately and I
    see
    there’s an error: “111 connection refused error” again and then it
    connecting to app02.

Can someone please, tell me why this is happening and how can I change
this?

Thanks,
Ravi

Posted at Nginx Forum: