How to cache a file loaded through Ajax - JavaScript (Node JS)

Question

I am working on an application where I am loading a pdf file through https.get() in node js, sending the data in chunks to the front end. It works pretty well, The reason I did it like this is that I sometimes load the pdf from a remote server, which causes a CORS issue, so I have written a simple proxy to handle it for me. It works well, but the problem is, every time I make a request from the front-end, the pdf is re-loaded and each time it loads it. What I want to do is, when the pdf file is loaded for the first time, I should cache it may be on the front end or backend and then load it from the cache each time the page is refreshed.

The proxy written in Node JS

The reading the file stream on front end

const url = "http://localhost:8003/pdf-cors";
const file = "https://veraviafit.com/wp-content/uploads/2018/11/N2RPlan-3-1.pdf";
fetch(`${url}?file=${file}`).then(async(response) => {
  const reader = response!.body!.getReader();
  const stream = new ReadableStream({
    start(controller) {
      // The following function handles each data chunk
      function push() {
        // "done" is a Boolean and value a "Uint8Array"
        reader.read().then((r) => {
          // console.log(r);
          const {
            done,
            value
          } = r;
          console.log("VALUE ---> ", value);
          // Is there no more data to read?
          if (done) {
            // Tell the browser that we have finished sending data
            controller.close();
            return;
          }
          // console.log(value);
          // Get the data and send it to the browser via the controller
          controller.enqueue(value);
          push();
        });
      };
      push();
    }
  });

  return new Response(stream, {
    headers: {
      "Content-Type": "text/html"
    }
  });
});

score 0 · Answer 1 · answered Nov 03 '20 at 10:55

0

Check out the memoize function from lodash. You could implement a function like this yourself quite easily if you don't want to include lodash.

https://lodash.com/docs/4.17.15#memoize

answered Nov 03 '20 at 10:55

Phil

307
3
12

I am aware of memoization, but what if the file is 100+ MBs would that work as expected? – Nadeem Ahmad Nov 03 '20 at 10:56
I misread your question slightly, sorry for the too brief answer. You'd need to store the files on your server in order to not have to refetch them every time from external sources. Memoizing the requested url could then help to figure out if it a new file, or if the file has been requested before. – Phil Nov 03 '20 at 11:05
Yeah, that will be useful in that case, but how can we do that on server side ? – Nadeem Ahmad Nov 03 '20 at 11:09

Nadeem Ahmad · Answer 2 · 2020-11-03T14:17:33.957

I found a feasible solution that dropped the loading time perfectly. So the first time it took 2.5 minutes to load this pdf (really depends on the internet speed), and the second time when I reloaded it, it took only 54 seconds to load the pdf as it was cached, all I had to do is to set response cache headers, which will actually cache the loaded data. You can visit this question to see how it works, I set it like this in my proxy code.

In order to load cache the dynamic files, I let the file load for the first time and save it with a specified cache 'key', and then write the whole chunks to that file. After the file is loaded, when I reload the page, and a request is made for the same file, I am sending back data from the file and not loading external resources on each call. It loads the file in milliseconds.

How to cache a file loaded through Ajax - JavaScript (Node JS)

2 Answers2