Intermittent SSL Handshake issues on Ubuntu 12.04 and Nginx

Hi Guys,

I posted originally my issue on askubuntu but I think this will be a
better
place

http://askubuntu.com/questions/611418/intermittent-ssl-handshake-issues-on-ubuntu-12-04-and-nginx.

Original post

In simple terms

I am having issues with https handshakes. I am currently using nginx but
it
is most likely not an nginx issue.

Behaviour

Web clients such as browsers will sometimes present “SSL connection
error”
(Chrome)

Apache benchmark will spit out several error lines and will report
around
1-10% failures. Errors below will appear in random order but the first
one
is more common.

(1) Benchmarking mysite.net (be patient)…SSL read failed (1) - closing
connection
128494120003296:error:1408F119:SSL routines:SSL3_GET_RECORD:decryption
failed or bad record mac:s3_pkt.c:486:

(2) SSL read failed (1) - closing connection
128494120003296:error:140943FC:SSL routines:SSL3_READ_BYTES:sslv3 alert
bad
record mac:s3_pkt.c:1262:SSL alert number 20

Server setup

Ubuntu:

Ubuntu 12.04 64bit with all updates and patches installed, server
restarted.
Nginx:

nginx/1.6.3 - from nginx.org (deb Index of /packages/ubuntu/
precise
nginx)

OpenSSL dynamically linked:

ldd which nginx | grep ssl

libssl.so.1.0.0 => /lib/x86_64-linux-gnu/libssl.so.1.0.0

(0x00007f3065569000)

strings /lib/x86_64-linux-gnu/libssl.so.1.0.0 | grep "^OpenSSL "

OpenSSL 1.0.1 14 Mar 2012

Nginx server config (with limited cyphers)
OpenSSL:

1.0.1 14 Mar 2012

#dpkg -s libssl1.0.0
Version: 1.0.1-4ubuntu5.25

#Workarounds

So far, the only workaround I found, is to narrow down available

cyphers.

Instead of using Mozilla Intermediate set, I found these would work
without
any issues:

ssl_ciphers
‘ECDHE-RSA-AES256-SHA384:ECDHE-RSA-AES128-SHA256:ECDHE-RSA-AES256-SHA:ECDHE-RSA-AES128-SHA:DHE-RSA-AES256-SHA256:DHE-RSA-AES128-SHA256:DHE-RSA-AES256-SHA:DHE-RSA-AES128-SHA:ECDHE-RSA-DES-CBC3-SHA:EDH-RSA-DES-CBC3-SHA:AES256-SHA256:AES128-SHA256:AES256-SHA:AES128-SHA:!aNULL:!eNULL:!EXPORT:!DES:!MD5:!PSK:!RC4’;

Second option is to downgrade to stock nginx (1.1.19-1ubuntu0.7)

#Things I tried

Because I am mainly using nginx as a proxy / load balancer, I tried

replacing nginx with HA-Proxy 1.5. Unfortunately I got the same problem.
I tried compiling nginx 1.6.3 with openssl 1.0.1m - no change.
On-line https/ssl validity tester did not found any issues with any
of
the certificates.
Disabling other nginx sites did not help either.

#Things I noticed

Interestingly this problem does not occur when using apache 

benchmark
from the server itself or it’s immediate neighbours, but it does happen
when
connecting from outside of the data centre. Apparently DC guys (coreix)
claim not to have any DDOS prevention system in front of the servers
which
would cause such an issue.
This issue is happening mainly on one of the https domains and is
very
sporadic for remaining two - hosted on the same box.

Posted at Nginx Forum:

Hello!

On Sun, Apr 19, 2015 at 06:08:35PM -0400, rPawel wrote:

In simple terms

1-10% failures. Errors below will appear in random order but the first one

OpenSSL dynamically linked:

1.0.1 14 Mar 2012

#dpkg -s libssl1.0.0
Version: 1.0.1-4ubuntu5.25

This looks similar to this ticket (turned out to be a bug in
OpenSSL, see comments for details):

http://trac.nginx.org/nginx/ticket/215

Try upgrading to OpenSSL 1.0.1h or newer to see if it helps.
Alternatively, make sure the OpenSSL package you are using
includes the fix in question.

[…]


Maxim D.
http://nginx.org/