HTTP: Generating ETag Header

Question

How do I generate an ETag HTTP header for a resource file?

score 19 · Answer 1 · answered Aug 09 '08 at 13:50

As long as it changes whenever the resource representation changes, how you produce it is completely up to you.

You should try to produce it in a way that additionally:

doesn't require you to re-compute it on each conditional GET, and
doesn't change if the resource content hasn't changed

Using hashes of content can cause you to fail at #1 if you don't store the computed hashes along with the files.

Using inode numbers can cause you to fail at #2 if you rearrange your filesystem or you serve content from multiple servers.

One mechanism that can work is to use something entirely content dependent such as a SHA-1 hash or a version string, computed and stored once whenever your resource content changes.

score 17 · Accepted Answer · edited Jul 26 '20 at 12:54

17

An etag is an arbitrary string that the server sends to the client that the client will send back to the server the next time the file is requested.

The etag should be computable on the server based on the file. Sort of like a checksum, but you might not want to checksum every file sending it out.

 server                client
 
        <------------- request file foo
 
 file foo etag: "xyz"  -------->
 
        <------------- request file foo
                       etag: "xyz" (what the server just sent)
 
 (the etag is the same, so the server can send a 304)

I built up a string in the format "datestamp-file size-file inode number". So, if a file is changed on the server after it has been served out to the client, the newly regenerated etag won't match if the client re-requests it.

char *mketag(char *s, struct stat *sb)
{
    sprintf(s, "%d-%d-%d", sb->st_mtime, sb->st_size, sb->st_ino);
    return s;
}

edited Jul 26 '20 at 12:54

Julian Reschke

40,156
8
95
98

answered Aug 07 '08 at 08:57

Mark Harrison

297,451
125
333
465

2

If mtime is the time the file was last changed, then what's the purpose of size and inode? – Steve Feb 04 '10 at 03:20
In my case, it's because it was a computed path from a CGI program. You're right that in the case of a direct path that the mtime would probably be sufficient. Since the cost is mainly going to be in stat(), there's no extra charge for including the inode and size, which might protect from the (quite unlikely, of course) case where a rogue admin might update a file and touch it back to the original mtime. – Mark Harrison Feb 04 '10 at 04:21
@MarkHarrison, why do you need the double quotes around the etag? Is it a mandated part of the syntax? – Pacerier Jul 18 '12 at 20:02

score 7 · Answer 3 · edited Jun 20 '20 at 09:12

7

From http://developer.yahoo.com/performance/rules.html#etags:

By default, both Apache and IIS embed data in the ETag that dramatically reduces the odds of the validity test succeeding on web sites with multiple servers.

...

If you're not taking advantage of the flexible validation model that ETags provide, it's better to just remove the ETag altogether.

edited Jun 20 '20 at 09:12

Community

1
1

answered Aug 07 '08 at 10:13

grom

15,842
19
64
67

score 2 · Answer 4 · answered Sep 05 '11 at 12:38

How to generate the default apache etag in bash

for file in *; do printf "%x-%x-%x\t$file\n" `stat -c%i $file` `stat -c%s $file` $((`stat -c%Y $file`*1000000)) ; done

Even when i was looking for something exactly like the etag (the browser asks for a file only if it has changed on the server), it never worked and i ended using a GET trick (adding a timestamp as a get argument to the js files).

score 1 · Answer 5 · answered Dec 27 '12 at 15:58

Ive been using Adler-32 as an html link shortener. Im not sure whether this is a good idea, but so far, I havent noticed any duplicates. It may work as a etag generator. And it should be faster then trying to hash using an encryption scheme like sha, but I havent verified this. The code I use is:

 shortlink = str(hex(zlib.adler32(link)+(2**32-1)/2))[2:-1]

score 0 · Answer 6 · answered Sep 19 '08 at 22:31

0

I would recommend not using them and going for last-modified headers instead.

Askapache has a useful article on this. (as they do pretty much everything it seems!)

http://www.askapache.com/htaccess/apache-speed-etags.html

answered Sep 19 '08 at 22:31

Rich Bradshaw

71,795
44
182
241

Hmm, that's a shame, hope they come back up soon as the site was a goldmine of advice! – Rich Bradshaw Feb 03 '10 at 20:20
@RichBradshaw just a question, How we can know the last modified timestamp of a resource? Is it totally a manual operation after going through db change history? Or are there any automated ways? – Supun Wijerathne Nov 28 '17 at 06:46

Sergey Ponomarev · Answer 7 · 2020-07-26T19:52:49.557

The code example of Mark Harrison is similar to what used in Apache 2.2. But such algorithm causes problems for load balancing when you have two servers with the same file but the file's inode is different. That's why in Apache 2.4 developers simplified ETag schema and removed the inode part. Also to make ETag shorter usually they encoded in hex:

    
<inttypes.h>
    
    
char *mketag(char *s, struct stat *sb)
{
    sprintf(s, "\"%" PRIx64 "-%" PRIx64 "\"", sb->st_mtime, sb->st_size);
    return s;
}

or for Java

 etag = '"' + Long.toHexString(lastModified) + '-' +
                                Long.toHexString(contentLength) + '"';

for C#

// Generate ETag from file's size and last modification time as unix timestamp in seconds from 1970
public static string MakeEtag(long lastMod, long size)
{
    string etag = '"' + lastMod.ToString("x") + '-' + size.ToString("x") + '"';
    return etag;
}

public static void Main(string[] args)
{
    long lastMod = 1578315296;
    long size = 1047;
    string etag = MakeEtag(lastMod, size);
    Console.WriteLine("ETag: " + etag);
    //=> ETag: "5e132e20-417"
}

The function returns ETag compatible with Nginx. See comparison of ETags form different servers

HTTP: Generating ETag Header

7 Answers7

Linked