Allow overriding etag function in application

[ashi009]

Current express calculates etag via utils.etag(), which is a private function, and could be overrided by:

1. res.set('etag', 'etag value') before res.send(), or
2. override by wrapping res.send() by setting res.set('etag', 'etag value') before forward arguments to the old res.send().

These two method works, but all come with catches:

1. require to call res.set() explicitly in every route, which is very inconvenient;
2. new res.send() must handle all supported method signatures, to generate etag correctly based on modified body. Which comes with performance penalty and may lead to forward compatibility issue.

So a more clean solution is to expose etag on application and response level, which allows override by doing:

app.etag = function(buf) {
  return '"' + crc32c.calculate(buf) + '"';
};

or

function strongEtag(buf) {
  return '"' + crypto.createHash('sha1').update(buf).digest('hex') + '"';
}

function strongEtagMiddleware(req, res, next) {
  res.etag = strongEtag;
  next();
}

app.get('/critical', strongEtagMiddleware, function(req, res) {
  res.send('critical data');
});

[dougwilson]

At least for the global ETag settings, I can just match proxy trust setting:

app.enable('etag') // use strong etags
app.set('etag', 'strong') // same
app.set('etag', 'weak') // weak etags
app.set('etag', function(chunk, encoding){ /* return valid etag */ }) // your own fn

[dougwilson]

I am also working with the fresh module so it supports weak ETags, if we are going to start producing them.

[ashi009]

Sounds great.

But using 'strong', 'weak' may cause confusion. As ETag: "abc" generated by a weak hash function is still a strong etag validator, and ETag: W/"489c152c8c35687dc90fd41f25d8d13f" generated by strong hash function is still a weak etag validator.

[dougwilson]

But using 'strong', 'weak' may cause confusion.

Those keywords relate to default hash functions that are either strong or weak; they mark the ETag appropriately according to their weakness. Do you have a better name? We only provide two: strong and weak, so I don't think there are any other ways to describe them.

[ashi009]

How about use hash algorithm's name?
On May 23, 2014 10:14 PM, "Douglas Christopher Wilson" <
notifications@github.com> wrote:

But using 'strong', 'weak' may cause confusion.

Those keywords relate to default hash functions that are either strong or
weak; they mark the ETag appropriately according to their weakness. Do you
have a better name? We only provide two: strong and weak, so I don't think
there are any other ways to describe them.

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/2129#issuecomment-44013516
.

[dougwilson]

How about use hash algorithm's name?

Because it is supposed to be invisible so we can change it at any time (we also want to just provide two: some generic strong one ans some generic weak one). If you wanted a specific hash algorithm, you would supply your own function.

[ashi009]

Cool, that works for me.

I have a little suggestion about the fresh module. Is it possible to export condition test method (not as middleware), so other people may not need to write their own? As sometimes, when serving some resources, I could hold the validator for the response, and I could test it against the condition in the request and short circuit to 304, instead of generating response again, hash it and test against the condition.

[dougwilson]

Is it possible to export condition test method

It should be available as req.fresh; you just have to set the ETag before you call it to be of any use (since it needs to know the request's outgoing ETag to know if it is fresh or not). Is this not what you are looking for? Perhaps an example can help me understand :)

[ashi009]

(since it needs to know the request's outgoing ETag to know if it is fresh or not). Is this not what you are looking for? Perhaps an example can help me understand :)

Sorry, I didn't make it clear.

Here is the thing, consider the response is large, like a huge data chunk from db. Here is how it works now:

app.get('/data', function(req, res) {
  db.findAll('some query', function(err, data) {
    if (err) return next(err);
    res.set('ETag', '"' + data.version + '"');
    res.json(data);
  });
});

And this data is rarely changed (but it may change), I could cache the data.version, and use that to check with req's if-not-match header, so I could send 304 without pulling the data:

app.get('/data', function(req, res) {
  if (req.headers['if-not-match'] && req.headers['if-not-match'] == cachedEtag)
    return res.send(304);
  db.findAll('some query', function(err, data) {
    if (err) return next(err);
    res.set('ETag', '"' + data.version + '"');
    res.json(data);
  });
});

As you see, my simplified conditional req handler is not complete. It could be nice to have one, so I could do something like:

app.get('/data', function(req, res) {
  if (req.matchConditon(cachedEtag, lastModify))
    return res.send(304);
  db.findAll('some query', function(err, data) {
    if (err) return next(err);
    res.set('ETag', '"' + data.version + '"');
    res.json(data);
  });
});

[dougwilson]

Ah. You can do that with:

var fresh = require('fresh')
app.get('/data', function(req, res) {
  cachedData = {
    etag: cachedEtag,
    'last-modified': lastModify
  }
  if (fresh(req.headers, cachedData))
    return res.send(304);
  db.findAll('some query', function(err, data) {
    if (err) return next(err);
    res.set('ETag', '"' + data.version + '"');
    res.json(data);
  });
});

You could simply cache your all your response headers, or just the two above in cachedData. If you want a different interface to the fresh module, you'll have to file an issue over there.

[ashi009]

My bad, I thought it was a middleware. Thanks