dynamically loading JSON file from javascript

Question

On my RPi I have an application (developed in C++) running in the background that does some complex mathematics based on some sensor input and produces some result every second. I want to display this data on a website. So I had the idea to have the application produce a JSON formatted file and read that interactively from a javascript script.

The app now produces a file ModelState.json in my html directory that looks like

{ "x" : -0.886289 , "y" : -0.434931 }

Based on this answer, I wrote the following html/js

<!doctype html>
<html>
<head>
    <meta charset="UTF-8">
    <title>Title</title>
</head>
<body>
    <p id="ModelState"></p>
</body>

<script type="text/javascript">
    window.onload = function () {
        setInterval(showModelState, 1000);

        function showModelState() {
            readJsonFile("ModelState.json", function(ModelStateJson){
                var ModelStateObj = JSON.parse(ModelStateJson);
                if (ModelStateObj.x && ModelStateObj.y) {
                    document.getElementById("ModelState").innerHTML =
                        "x: " + ModelStateObj.x + ", " +
                        "y: " + ModelStateObj.y;
                }
            });
        }
        function readJsonFile(file, callback) {
            let rawFile = new XMLHttpRequest();
            rawFile.overrideMimeType("application/json");
            rawFile.open("GET", file, true);
            rawFile.onreadystatechange = function() {
                if (rawFile.readyState === 4 && rawFile.status === 200) {
                    callback(rawFile.responseText);
                }
            }
            rawFile.send(null);
        }
    }
</script>
</html>

However, what I observe is that the file seems to be loaded once. The data on the webpage doesn't change, while the data in the file does change. I don't know why. Is it that the XMLHttpRequest keeps the file open, such that onreadystatechange is not triggered and the callback function is not called again? I would expect the send response to finish the request, thus close the file.

I guess the browser is caching the response, what if you add a timestamp to file e.g.: `\`ModelState.json?time=${Date.now()}\`` for cache bursting purposes. — cristobal, Jul 07 '20 at 12:01
Declare `readJsonFile` outside of `showModelState`. Why are you declaring a function inside another one that's being called every second? Just declare it once and for all, don't redeclare it every second — Jeremy Thille, Jul 07 '20 at 12:09
@JHBonarius the browser will automatically cache files for a given time, that depends on the browser and their internal defaults for files that do not send any cache headers. The problem here is that since the filename does not change, the browser thinks that file has not changed since last fetch, and should just return the data previously fetched. — cristobal, Jul 07 '20 at 12:40
@cristobal Thanks for pointing me in the right direction. I also found that browsers don't cache POST requests, so changing the method to "POST" also solves the issue. — JHBonarius, Jul 07 '20 at 12:45
Is the file served locally or do you have an server serving the `ModelState.json` file? If you have an server you could check from cache headers sent from developers tools if the browser has once or via the commandline e.g. `curl -IL -X GET http(s)://.../ModelState.json`. — cristobal, Jul 07 '20 at 12:46
If you have an server serving the ModelState.json file, you could tune the server to set the cache header to `no-cache, max-age=0`. https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Cache-Control — cristobal, Jul 07 '20 at 12:49

JHBonarius · Accepted Answer · 2020-07-08T09:32:35.783

As cristobal states in the comments, the problem is caused by the browser caching the read file.

One solution is to confuse the browser with a dynamic url, e.g. by changing

readJsonFile("ModelState.json", function(...

to

readJsonFile("ModelState.json?time=${Date.now()}", function(...

An alternative solution (based on this) is that the browser doesn't cache POST http request methods. Thus you can change

rawFile.open("GET", file, true);

to

rawFile.open("POST", file, true);

A third solution (found here) is to add a pragma after the line rawFile.open("GET"...

rawFile.setRequestHeader("Pragma", "no-cache");

More elaborate request headers for cache control can be found here

dynamically loading JSON file from javascript

1 Answers1