re-require node file after writing to it with fs

Question

Inside a node server, I am fetching data (as a string) that gets written to local file, after which I want to use the updated data inside my main function. After looking through solutions that delete the require.cache for that file and re-requiring it, the previous information in the file is still being cached -- meaning that if it was overwritten with new data, it still references the old, and if the file was blank before being overwritten, it will return undefined.

How can I force a re-evaluation of the data at runtime within my main function?

What I'm currently doing is require at the top:

let d_file = require("path_to_d_file.js")

then function for fetching data and writing it to the file:

const async_write = async () => {
    const new_data = await axios.get(url, headers);
    fs.writeFile("path_to_d_file.js", new_data.data, (err) => {
        if (err) return console.error(err);
    }
}

Execute this function and attempt to re-require data after clearing the cache:

const main_func = async () => {
    await async_write();
    delete require.cache[require.resolve("path_to_d_file")];
    d_file = require("path_to_d_file");

    d_file.stuff.map(() => <do things>)
}

But d_file.stuff is undefined if the file was previously empty, or the older data if the file was not previously empty. I even tried deleting the cache inside the callback of fs.writeFile and re-assigning the import variable there without success. I also attempted a lazy require instead of declaring it at the top and re-assigning as seen here: https://stackoverflow.com/a/37083469/11925070 with similar results.

Is there a way to do what I'm attempting to do here without reinventing the wheel?

score 0 · Answer 1 · answered Jun 21 '21 at 20:07

fs.writeFile is an asynchronous function and will take some milliseconds to finish the file writing. Since your code is not waiting for the callback of this function to return, the require is being called before the file write operation finishes.

You can either use fs.writeFileSync or wait for the callback:

// ...
const new_data = await axios.get(url, headers);
await new Promise((resolve, reject) => {
    fs.writeFile("./path_to_d_file.js", new_data.data, (err) => {
        if (err) return reject(err);
        resolve();
    });
});
// ...

And you also can use Dynamic import() instead of require:

d_file = (await import("./path_to_d_file.js")).default;

re-require node file after writing to it with fs

1 Answers1