Using Content-MD5 header

I would like to use the Content-MD5 header from a HTTP response, does
Nginx provide a way to add the ‘Content-MD5’ header to the response for
a static file request?

  • Matthijs L.

On Mon, Jul 21, 2008 at 01:37:56PM +0200, Matthijs L. wrote:

I would like to use the Content-MD5 header from a HTTP response, does
Nginx provide a way to add the ‘Content-MD5’ header to the response for
a static file request?

No, nginx does not support Content-MD5 header.
Calculating MD5 means that nginx must read whole file before sending it
to client.

What purpose of this header ?

security…

you can ensure that the file is not changed, modified, damaged

might be implemented that the md5 sums generated not on the fly but
after a restart for example you can enable it on a directory.

regards,
Istvan

Istvan Szukacs ha scritto:

Igor S. wrote:

On Mon, Jul 21, 2008 at 01:37:56PM +0200, Matthijs L. wrote:

No, nginx does not support Content-MD5 header.
Calculating MD5 means that nginx must read whole file before sending it
to client.

What purpose of this header ?

security…

you can ensure that the file is not changed, modified, damaged

might be implemented that the md5 sums generated not on the fly but
after a restart for example you can enable it on a directory.

regards,
Istvan

This can be easily and efficiently implemented by writing a module for
Nginx.

As an example:

  1. The module is registered for the access phase
  2. For each request the module opens a DBM file
    (whose path is in configuration).
    In the dbm file are stored path names and MD5 hashes.
    The module check if the path name is in the database and retrieve
    the MD5 hash, adding the MD5 header.
    If the path is not in the database, an error is returned.
  3. Control then pass to the static module, and file is served to the
    client.

The DBM file is updated using an external tool (as an example a Python
or Perl script): it is important that the dbm file is updated
atomically, and this can be done by creating a temporary file and then
renaming it (since renaming is an atomical operation).

Manlio P.

Manlio P. wrote:

As an example:

  1. The module is registered for the access phase

Could you please tell us more about phases? I’m not sure I understand
this
subject clearly.

Valery K. ha scritto:

Manlio P. wrote:

As an example:

  1. The module is registered for the access phase

Could you please tell us more about phases? I’m not sure I understand this
subject clearly.

Each HTTP request is handled via successive phases.
From src/ngx_http_core_module.h:

typedef enum {
NGX_HTTP_POST_READ_PHASE = 0,

 NGX_HTTP_SERVER_REWRITE_PHASE,

 NGX_HTTP_FIND_CONFIG_PHASE,
 NGX_HTTP_REWRITE_PHASE,
 NGX_HTTP_POST_REWRITE_PHASE,

 NGX_HTTP_PREACCESS_PHASE,

 NGX_HTTP_ACCESS_PHASE,
 NGX_HTTP_POST_ACCESS_PHASE,

 NGX_HTTP_CONTENT_PHASE,

 NGX_HTTP_LOG_PHASE

} ngx_http_phases;

The ngx_http_static_module is executed at NGX_HTTP_CONTENT_PHASE.

Manlio P.

Manlio P. wrote:

 NGX_HTTP_POST_REWRITE_PHASE,

 NGX_HTTP_PREACCESS_PHASE,

 NGX_HTTP_ACCESS_PHASE,
 NGX_HTTP_POST_ACCESS_PHASE,

 NGX_HTTP_CONTENT_PHASE,

 NGX_HTTP_LOG_PHASE

} ngx_http_phases;

I have certainly read this code and I can generally guess what it is
supposed to mean. But to write production code I would like to know what
is being performed at each of the phases and what the handler at each
phase can do with the request.

Matthijs L. ha scritto:

What purpose of this header ?

Igor,

I’m writing an app that handles file uploads (assets storage) from other
machines in the network, mostly done by batch processing. I’m using a
HEAD request to check if a file already is uploaded, but it’s hard to
monitor which files already have been edited and need to be uploaded
again.

Can’t you use the Last-Modified header?

[…]

Manlio P.

Igor S. wrote:

On Mon, Jul 21, 2008 at 01:37:56PM +0200, Matthijs L. wrote:

I would like to use the Content-MD5 header from a HTTP response, does
Nginx provide a way to add the ‘Content-MD5’ header to the response for
a static file request?

No, nginx does not support Content-MD5 header.
Calculating MD5 means that nginx must read whole file before sending it
to client.

What purpose of this header ?

Igor,

I’m writing an app that handles file uploads (assets storage) from other
machines in the network, mostly done by batch processing. I’m using a
HEAD request to check if a file already is uploaded, but it’s hard to
monitor which files already have been edited and need to be uploaded
again.
By using the Content-MD5 header and a locally calculated MD5 checksum, I
can easily check if a file is outdated.
Ofcourse there are other solutions to this problem:
I could also do a GET request to http://some/image.gif/md5, to hit an
app which calculates the MD5 on request, so nginx doesn’t need to
calculate the MD5 all the time, even for requests that don’t use it.
Or I could prepend the filename with (a part) of the MD5 hash. This
means that I could do a cheap HEAD to http://some/a34sdfs3-image.gif to
see if my image already exists.

Would it be expensive to add a md5 calculation for static files?

Valery K. ha scritto:

 NGX_HTTP_FIND_CONFIG_PHASE,
 NGX_HTTP_LOG_PHASE

} ngx_http_phases;

I have certainly read this code and I can generally guess what it is
supposed to mean. But to write production code I would like to know what
is being performed at each of the phases and what the handler at each
phase can do with the request.

All I have done is read some Nginx modules.
You can grep these enum names inside the Nginx source code.

I only know what it is done in PREACCESS, ACCESS, CONTENT and LOG
phases, since the modules http_realip, http_access, index_module and
log_module are quite simple and it is where you may want to run your
module

Manlio P.