答复: nginx plus with ssl on TCP load balance not work

Server NGINX
1

With info level log enabled.

Found these:

80’s log:
2015/06/11 08:48:18 [info] 12719#0: *449 client 10.0.0.1:1494 connected
to 0.0.0.0:80
2015/06/11 08:48:18 [info] 12719#0: *449 proxy 172.31.5.228:17019
connected to 10.0.0.2:80
2015/06/11 08:48:19 [info] 12719#0: *449 upstream disconnected, bytes
from/to client:689/7900, bytes from/to upstream:7900/689

It’s success

443’s log: tried several times, not work, now page show
ERR_CONNECTION_CLOSED, still not work

2015/06/11 08:48:28 [info] 12719#0: *451 client 10.0.0.1:1642 connected
to 0.0.0.0:80
2015/06/11 08:48:28 [info] 12719#0: *451 proxy 172.31.5.228:26620
connected to 10.0.0.3:80
2015/06/11 08:48:28 [info] 12719#0: *451 upstream disconnected, bytes
from/to client:704/452, bytes from/to upstream:452/704
2015/06/11 08:48:28 [info] 12719#0: *453 client 10.0.0.1:1518 connected
to 0.0.0.0:443
2015/06/11 08:48:28 [info] 12719#0: *453 proxy 172.31.5.228:17021
connected to 10.0.0.2:80
2015/06/11 08:48:28 [info] 12719#0: *453 upstream disconnected, bytes
from/to client:517/0, bytes from/to upstream:0/517
2015/06/11 08:48:28 [info] 12719#0: *455 client 10.0.0.1:2943 connected
to 0.0.0.0:443
2015/06/11 08:48:28 [info] 12719#0: *455 proxy 172.31.5.228:26622
connected to 10.0.0.3:80
2015/06/11 08:48:28 [info] 12719#0: *455 upstream disconnected, bytes
from/to client:221/0, bytes from/to upstream:0/221
2015/06/11 08:48:28 [info] 12719#0: *457 client 10.0.0.1:2187 connected
to 0.0.0.0:443
2015/06/11 08:48:28 [info] 12719#0: *457 proxy 172.31.5.228:17023
connected to 10.0.0.2:80
2015/06/11 08:48:28 [info] 12719#0: *457 upstream disconnected, bytes
from/to client:174/0, bytes from/to upstream:0/174
2015/06/11 08:48:28 [info] 12719#0: *459 client 10.0.0.1:2346 connected
to 0.0.0.0:443
2015/06/11 08:48:28 [info] 12719#0: *459 proxy 172.31.5.228:26624
connected to 10.0.0.3:80
2015/06/11 08:48:28 [info] 12719#0: *459 upstream disconnected, bytes
from/to client:174/0, bytes from/to upstream:0/174
2015/06/11 08:48:29 [info] 12719#0: *461 client 10.0.0.1:2495 connected
to 0.0.0.0:443
2015/06/11 08:48:29 [info] 12719#0: *461 proxy 172.31.5.228:17025
connected to 10.0.0.2:80
2015/06/11 08:48:29 [info] 12719#0: *461 upstream disconnected, bytes
from/to client:517/0, bytes from/to upstream:0/517
2015/06/11 08:48:29 [info] 12719#0: *463 client 10.0.0.1:3742 connected
to 0.0.0.0:443
2015/06/11 08:48:29 [info] 12719#0: *463 proxy 172.31.5.228:26626
connected to 10.0.0.3:80
2015/06/11 08:48:29 [info] 12719#0: *463 upstream disconnected, bytes
from/to client:221/0, bytes from/to upstream:0/221
2015/06/11 08:48:29 [info] 12719#0: *465 client 10.0.0.1:3743 connected
to 0.0.0.0:443
2015/06/11 08:48:29 [info] 12719#0: *465 proxy 172.31.5.228:17027
connected to 10.0.0.2:80
2015/06/11 08:48:29 [info] 12719#0: *465 upstream disconnected, bytes
from/to client:174/0, bytes from/to upstream:0/174
2015/06/11 08:48:29 [info] 12719#0: *467 client 10.0.0.1:2343 connected
to 0.0.0.0:443
2015/06/11 08:48:29 [info] 12719#0: *467 proxy 172.31.5.228:26628
connected to 10.0.0.3:80
2015/06/11 08:48:29 [info] 12719#0: *467 upstream disconnected, bytes
from/to client:174/0, bytes from/to upstream:0/174

And from the backend web servers, found request not correct:
10.0.0.1,[11/Jun/2015:08:57:42
+0000],\x16\x03\x01\x02,/,HTTP/0.9,501,0,2030,-, 10.0.0.1

Normal request should be
172.31.11.248,[11/Jun/2015:09:00:30
+0000],GET,/signin,HTTP/1.1,200,5924,211592,Mozilla/5.0 (Windows NT 6.1;
WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/42.0.2311.60
Safari/537.36,36.7.69.39, 172.31.11.248

So it that any bug?

-----邮件原件-----
发件人: smith [mailto:[email protected]]
发送时间: 2015年6月11日 8:35
收件人: ‘[email protected]’
主题: 答复: nginx plus with ssl on TCP load balance not work

When I’m trying http ssl, I found need to set proxy_set_header
X-Forwarded-Proto $scheme; in server block, or it will also encounter
ERR_TOO_MANY_REDIRECTS.

Is TCP has same kind of setting?

-----邮件原件-----
发件人: smith [mailto:[email protected]]
发送时间: 2015年6月11日 8:28
收件人: [email protected]
主题: 答复: nginx plus with ssl on TCP load balance not work

The 80 is normal, And I tried use http ssl, also works. Don’t know Why
TCP not work.

-----邮件原件-----
发件人: [email protected] [mailto:[email protected]] 代表 Roman
Arutyunyan
发送时间: 2015年6月11日 8:25
收件人: [email protected]
主题: Re: nginx plus with ssl on TCP load balance not work

What about the 80 port of the stream balancer?
Does it proxy the connection normally?

PS: no access log is supported in the stream module.
Connection information (addresses etc) is logged to error log with the
info loglevel.

On 11 Jun 2015, at 10:49, smith [email protected] wrote:

events {
’
#gzip on;
And the content in previous email is in xxxx.d/xxxx.conf
Arutyunyan

Hi,
Connection timed out) while connecting to upstream, client: 10.0.0.1,
proxy_pass backend;
ssl_certificate ssl/chained.crt;

And I’m using amazon linux

nginx mailing list
[email protected]
nginx Info Page

nginx mailing list
[email protected]
nginx Info Page

–
Roman A.

nginx mailing list
[email protected]
http://mailman.nginx.org/mailman/listinfo/nginx

2

On Thu, Jun 11, 2015 at 09:03:55AM -0000, smith wrote:

2015/06/11 08:48:28 [info] 12719#0: *455 upstream disconnected, bytes from/to
client:221/0, bytes from/to upstream:0/221
2015/06/11 08:48:29 [info] 12719#0: *463 proxy 172.31.5.228:26626 connected to
10.0.0.3:80
10.0.0.1,[11/Jun/2015:08:57:42 +0000],\x16\x03\x01\x02,/,HTTP/0.9,501,0,2030,-,
10.0.0.1
主题: 答复: nginx plus with ssl on TCP load balance not work

http {
sendfile on;
stream {

server {
#ssl_session_cache shared:SSL:20m;
#ssl_session_timeout 4h;

}

upstream backend {
server ...:80;
server ...:80;
}

It looks like you have “proxy_ssl on;” in the stream{} block,
do you?

3

No, I did not set proxy_ssl on, is that default on?

Posted at Nginx Forum:

4

Any help on this? not working

Posted at Nginx Forum:

5

No, I did not set proxy_ssl on.

Sorry, mymistake, the log from the backend server is normal, but all of
them are 302, not 200. So there are always redirect, why?

10.0.0.2,[11/Jun/2015:14:34:33
+0000],GET,/signin,HTTP/1.1,302,0,12690,Mozilla/5.0 (Windows NT 6.1;
WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/43.0.2357.124
Safari/537.36, 10.0.0.2
10.0.0.2,[11/Jun/2015:14:34:33
+0000],GET,/signin,HTTP/1.1,302,0,12690,Mozilla/5.0 (Windows NT 6.1;
WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/43.0.2357.124
Safari/537.36, 10.0.0.2
10.0.0.2,[11/Jun/2015:14:34:33
+0000],GET,/signin,HTTP/1.1,302,0,12690,Mozilla/5.0 (Windows NT 6.1;
WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/43.0.2357.124
Safari/537.36, 10.0.0.2
10.0.0.2,[11/Jun/2015:14:34:33
+0000],GET,/signin,HTTP/1.1,302,0,12690,Mozilla/5.0 (Windows NT 6.1;
WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/43.0.2357.124
Safari/537.36, 10.0.0.2
10.0.0.2,[11/Jun/2015:14:34:33
+0000],GET,/signin,HTTP/1.1,302,0,12690,Mozilla/5.0 (Windows NT 6.1;
WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/43.0.2357.124
Safari/537.36, 10.0.0.2
10.0.0.2,[11/Jun/2015:14:34:33
+0000],GET,/signin,HTTP/1.1,302,0,12690,Mozilla/5.0 (Windows NT 6.1;
WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/43.0.2357.124
Safari/537.36, 10.0.0.2

And the log from nginx is still many try in nginx
2015/06/11 08:48:28 [info] 12719#0: *451 client 10.0.0.1:1642 connected
to 0.0.0.0:80
2015/06/11 08:48:28 [info] 12719#0: *451 proxy 172.31.5.228:26620
connected to 10.0.0.3:80
2015/06/11 08:48:28 [info] 12719#0: *451 upstream disconnected, bytes
from/to client:704/452, bytes from/to upstream:452/704
2015/06/11 08:48:28 [info] 12719#0: *453 client 10.0.0.1:1518 connected
to 0.0.0.0:443
2015/06/11 08:48:28 [info] 12719#0: *453 proxy 172.31.5.228:17021
connected to 10.0.0.2:80
2015/06/11 08:48:28 [info] 12719#0: *453 upstream disconnected, bytes
from/to client:517/0, bytes from/to upstream:0/517
2015/06/11 08:48:28 [info] 12719#0: *455 client 10.0.0.1:2943 connected
to 0.0.0.0:443
2015/06/11 08:48:28 [info] 12719#0: *455 proxy 172.31.5.228:26622
connected to 10.0.0.3:80
2015/06/11 08:48:28 [info] 12719#0: *455 upstream disconnected, bytes
from/to client:221/0, bytes from/to upstream:0/221
2015/06/11 08:48:28 [info] 12719#0: *457 client 10.0.0.1:2187 connected
to 0.0.0.0:443
2015/06/11 08:48:28 [info] 12719#0: *457 proxy 172.31.5.228:17023
connected to 10.0.0.2:80
2015/06/11 08:48:28 [info] 12719#0: *457 upstream disconnected, bytes
from/to client:174/0, bytes from/to upstream:0/174
2015/06/11 08:48:28 [info] 12719#0: *459 client 10.0.0.1:2346 connected
to 0.0.0.0:443
2015/06/11 08:48:28 [info] 12719#0: *459 proxy 172.31.5.228:26624
connected to 10.0.0.3:80
2015/06/11 08:48:28 [info] 12719#0: *459 upstream disconnected, bytes
from/to client:174/0, bytes from/to upstream:0/174
2015/06/11 08:48:29 [info] 12719#0: *461 client 10.0.0.1:2495 connected
to 0.0.0.0:443
2015/06/11 08:48:29 [info] 12719#0: *461 proxy 172.31.5.228:17025
connected to 10.0.0.2:80
2015/06/11 08:48:29 [info] 12719#0: *461 upstream disconnected, bytes
from/to client:517/0, bytes from/to upstream:0/517
2015/06/11 08:48:29 [info] 12719#0: *463 client 10.0.0.1:3742 connected
to 0.0.0.0:443
2015/06/11 08:48:29 [info] 12719#0: *463 proxy 172.31.5.228:26626
connected to 10.0.0.3:80
2015/06/11 08:48:29 [info] 12719#0: *463 upstream disconnected, bytes
from/to client:221/0, bytes from/to upstream:0/221
2015/06/11 08:48:29 [info] 12719#0: *465 client 10.0.0.1:3743 connected
to 0.0.0.0:443
2015/06/11 08:48:29 [info] 12719#0: *465 proxy 172.31.5.228:17027
connected to 10.0.0.2:80
2015/06/11 08:48:29 [info] 12719#0: *465 upstream disconnected, bytes
from/to client:174/0, bytes from/to upstream:0/174
2015/06/11 08:48:29 [info] 12719#0: *467 client 10.0.0.1:2343 connected
to 0.0.0.0:443
2015/06/11 08:48:29 [info] 12719#0: *467 proxy 172.31.5.228:26628
connected to 10.0.0.3:80
2015/06/11 08:48:29 [info] 12719#0: *467 upstream disconnected, bytes
from/to client:174/0, bytes from/to upstream:0/174

-----邮件原件-----
发件人: [email protected] [mailto:[email protected]] 代表 Ruslan
Ermilov
发送时间: 2015年6月11日 10:11
收件人: [email protected]
主题: Re: 答复: nginx plus with ssl on TCP load balance not work

On Thu, Jun 11, 2015 at 09:03:55AM -0000, smith wrote:

from/to client:689/7900, bytes from/to upstream:7900/689
2015/06/11 08:48:28 [info] 12719#0: *451 upstream disconnected, bytes
connected to 10.0.0.3:80
2015/06/11 08:48:28 [info] 12719#0: *459 proxy 172.31.5.228:26624
connected to 0.0.0.0:443
2015/06/11 08:48:29 [info] 12719#0: *467 client 10.0.0.1:2343

发送时间: 2015年6月11日 8:35
收件人: [email protected]

user nginx;
}

real-ip) I have web servers behind, I want to use ssl offloading, and

}
#ssl_ciphers HIGH:!aNULL:!MD5;
}
It looks like you have “proxy_ssl on;” in the stream{} block,
do you?

nginx mailing list
[email protected]
http://mailman.nginx.org/mailman/listinfo/nginx

6

As long as you get something from your backend, I don’t see anything
could
be wrong on the proxy. Backend get the connection, that’s all.

I guess it was because your backend get frustrated for all incoming
requests being from frontend. HTTP proxy can pass client IP info by
“X-Forwarded-For”. That doesn’t work for TCP proxy.

For testing, I suggest you to start with some basic http service on your
backend.

7

If you proxy http with a tcp proxy to an http backend, and receive the
302 code,
then IMHO you should look for problems in your http backend.

On 14 Jun 2015, at 03:53, huakaibird [email protected] wrote:

Any help on this? not working

Posted at Nginx Forum:
Re: 答复: 答复: nginx plus with ssl on TCP load balance not work

nginx mailing list
[email protected]
nginx Info Page

–
Roman A.

8

Thanks, is this means TCP proxy not suitable for web usage, for web
usage may have many redirect usage?

I’m going to use http/https proxy instead

发件人: [email protected] [mailto:[email protected]] 代表 ryd994
发送时间: 2015年6月15日 4:21
收件人: [email protected]
主题: Re: 答复: 答复: nginx plus with ssl on TCP load balance not work

As long as you get something from your backend, I don’t see anything
could be wrong on the proxy. Backend get the connection, that’s all.

I guess it was because your backend get frustrated for all incoming
requests being from frontend. HTTP proxy can pass client IP info by
“X-Forwarded-For”. That doesn’t work for TCP proxy.

For testing, I suggest you to start with some basic http service on your
backend.

On Sat, Jun 13, 2015 at 8:53 PM huakaibird [email protected] wrote:

Any help on this? not working

Posted at Nginx Forum:

9

My backend is work normally under nginx http/https load balancer.

And also works under amazon Elastic load balancer.

-----邮件原件-----
发件人: [email protected] [mailto:[email protected]] 代表 Roman
Arutyunyan
发送时间: 2015年6月15日 9:50
收件人: [email protected]
主题: Re: nginx plus with ssl on TCP load balance not work

Redirect usage is not related to the TCP proxy.
Please search for problems in your backend.

TCP proxy can be used for proxying any protocol unless you need to
change the data bytes (alter HTTP headers, change method etc)

主题: Re: 答复: 答复: nginx plus with ssl on TCP load balance not work

Posted at Nginx Forum:
Re: 答复: 答复: nginx plus with ssl on TCP load balance not work

nginx mailing list
[email protected]
nginx Info Page

nginx mailing list
[email protected]
nginx Info Page

–
Roman A.

nginx mailing list
[email protected]
http://mailman.nginx.org/mailman/listinfo/nginx

10

Redirect usage is not related to the TCP proxy.
Please search for problems in your backend.

TCP proxy can be used for proxying any protocol unless
you need to change the data bytes (alter HTTP headers,
change method etc)

nginx mailing list
[email protected]
nginx Info Page

nginx mailing list
[email protected]
nginx Info Page

–
Roman A.