Node.js: how to request an image only if it changed

Question

I'm designing a node.js app.
One of its tasks is to regularly download a set of images from some public, external, site.
One requirement is to avoid repeating the download of images which are not changed from the previous download.
I plan to use "request" module, since it is far more complete and flexible with respect to other networking modules (please correct me if I'm wrong).

This is the code I'm using now (please ignore some mistakes, like comparing dates with > or < operators, consider it pseudo-code...):

var request = require('request');
var myResource = {
  'url': 'http://www.example.com/image1.jpg',
  'last-modified': 'Mon, 28 Sep 2015 08:44:06 GMT'
};

request(
  myResource.url,
  { method: 'HEAD'},
  function (err, res, body) {
    if (err) {
      return console.error('error requesting header:', err);
    }
    var lastModifiedDate = res.headers['last-modified'];
    console.log('last modified date:', lastModifiedDate);
    if (lastModifiedDate > myResource['last-modified']) { // resource did change
      request(
        myResource.url,
        function (err, response, contents) {
          if (err) {
            return console.error('error requesting content:', err);
          }
          myResource['last-modified'] = lastModifiedDate;
          storeContents(contents); // store contents to DB
        }
      );
    }
  }
);

This code should work (in principle).
But I ask: request() is called twice: is this a waste of resources?
Could the content request be someway chained to the first request?
Can you suggest a cleaner / smarter / faster approach?

score 3 · Accepted Answer · edited May 23 '17 at 12:29

3

Maybe i'm missing something, but if you know the last-modified date, you should send that as the If-Modified-Since header with the GET request and skip the HEAD request. The server should return a 304 when appropriate.

How "304 Not Modified" works?

edited May 23 '17 at 12:29

Community

1
1

answered Sep 29 '15 at 21:41

Josh C.

4,303
5
30
51

Thanks! I didn't remember about `If-Modified-Since` header... This is probably **the** answer... I'm accepting it soon :-) – MarcoS Sep 29 '15 at 21:51
@MarcoS if not, try `ETag`, but either way you should leverage the image server and response header and use only one request. – Josh C. Sep 29 '15 at 21:55
How can I trigger the resource download based on ETag? Does something like `If-Etag-different` exists? – MarcoS Sep 29 '15 at 22:07
1

@MarcoS, check the link in my post. – Josh C. Sep 29 '15 at 22:18

Node.js: how to request an image only if it changed

1 Answers1