Nginx 0.7.59 open_file_cache and NFS

Server NGINX
1

Hello.

We have two nginx serving static objects from a shared NFS partition,
and we are using open_file_cache for performance purposes:

With nginx 0.6.x, when the developers uploads files, during a few
seconds (less than 30s) we see this errors in logs:

[alert] 2721#0: *75303 sendfile() failed (116: Stale NFS file handle)
while sending response to client, client: XXX …

I understand that is a normal behaviour because we are using a
open_file_cache.

But, when we upgrade to nginx 0.7.59 the same errors appears
indefinitely (and of course we are sending corrupted objects or
blank), it seems that, open_file_cache_valid is been ignored in new
version.

Any idea, tip or recomendation ?

In other hand, is recomended the use of open_file_cache, until now, we
are very happy with it.

Thanks in advance.
BR.

2

On Wed, Jun 10, 2009 at 11:49:00AM +0200, andan andan wrote:

But, when we upgrade to nginx 0.7.59 the same errors appears
indefinitely (and of course we are sending corrupted objects or
blank), it seems that, open_file_cache_valid is been ignored in new
version.

Any idea, tip or recomendation ?

In other hand, is recomended the use of open_file_cache, until now, we
are very happy with it.

What OS do you using on NFS client and server side ?
How files are uploaded - simple cp or

cp file file.temp
rm file
mv file.temp file

?

I believe in later case NFS should provide atomic operation.

3

2009/6/10 Igor S. [email protected]:

are very happy with it.

What OS do you using on NFS client and server side ?

Client: RHEL5
2.6.18-128.1.10.el5 x86_64

Server: NAS EMC Celerra NS40

How files are uploaded - simple cp or

cp file file.temp
rm file
mv file.temp file

Normally, they uploads files via scp, but editing a file by hand the
errors appears on the same way.

I believe in later case NFS should provide atomic operation.

Is this related to async/sync mount options ?

Thanks you Igor.
BR.

4

2009/6/10 Igor S. [email protected]:

Any idea, tip or recomendation ?

On local filesystem you may get mixed content from old and new versions
without any errors.

mmm, then, is recommended to disable open_file_cache in this scenario ?

I’m curious about the question that with nginx 0.6.x in a few seconds
the file is “ready” again, but no with 0.7.x

Thanks again.

5

On Wed, Jun 10, 2009 at 12:24:02PM +0200, andan andan wrote:

while sending response to client, client: XXX …
In other hand, is recomended the use of open_file_cache, until now, we

cp file file.temp
rm file
mv file.temp file

Normally, they uploads files via scp, but editing a file by hand the
errors appears on the same way.

This is because it’s no atomic method.
On local filesystem you may get mixed content from old and new versions
without any errors.

I believe in later case NFS should provide atomic operation.

Is this related to async/sync mount options ?

No.

6

On Wed, Jun 10, 2009 at 01:06:32PM +0200, andan andan wrote:

seconds (less than 30s) we see this errors in logs:

Server: NAS EMC Celerra NS40
This is because it’s no atomic method.
On local filesystem you may get mixed content from old and new versions
without any errors.

mmm, then, is recommended to disable open_file_cache in this scenario ?

Even if you will disable open_file_cache, the race condition still
remains.
open_file_cache simply makes it distinct: 30s vs 20ms and so.

I’m curious about the question that with nginx 0.6.x in a few seconds
the file is “ready” again, but no with 0.7.x

Can not say right now.

7

2009/6/10 Igor S. [email protected]:

blank), it seems that, open_file_cache_valid is been ignored in new
2.6.18-128.1.10.el5 x86_64
errors appears on the same way.
I’m curious about the question that with nginx 0.6.x in a few seconds
the file is “ready” again, but no with 0.7.x

Can not say right now.

Thanks so much for your feedback is very apreciated.

BR.