1

I have a NodeJS server managing some files. It's going to watch for a known filename from an external process and, once received, read it and then delete it. However, sometimes it's attempted to be read/deleted before the file has "unlocked" from previous use so likely will fail occasionally. What I'd like to do is retry this file asap, either as soon as it's finished or continuously at a fast pace.

I'd rather avoid a long sleep where possible, because this needs to be handled ASAP and every second counts.

fs.watchFile(intput_json_file, {interval: 10}, function(current_stats, previous_stats) {

                var json_data = "";
                try {
                    var file_cont = fs.readFileSync(input_json_file); // < TODO: async this
                    json_data = JSON.parse(file_cont.toString());
                    fs.unlink(input_json_file);
                } catch (error) {
                    console.log("The JSON in the could not be parsed. File will continue to be watched.");
                    console.log(error);
                    return;
                }

                // Else, this has loaded properly.
                fs.unwatchFile(input_json_file);

                // ... do other things with the file's data.

}


// set a timeout for the file watching, just in case
setTimeout(fs.unwatchFile, CLEANUP_TIMEOUT, input_json_file);

I expect "EBUSY: resource busy or locked" to turn up occasionally, but fs.watchFile isn't always called when the file is unlocked.

I thought of creating a function and then calling it with a delay of 1-10ms, where it could call itself if that fails too, but that feels like a fast route to a... cough stack overflow.

I'd also like to steer clear of synchronous methods so that this scales nicely, but being relatively new to NodeJS all the callbacks are starting to turn into a maze.

IamPancakeMan
  • 199
  • 1
  • 10
  • Can you alter the external service that creates the file? – Heretic Monkey May 29 '19 at 14:57
  • 1
    Otherwise, you could use a polling method like that in [Node Js: Test to see if a file is locked for editing by another process](https://stackoverflow.com/q/37690321/215552) – Heretic Monkey May 29 '19 at 15:00
  • @HereticMonkey I'd prefer not to because it would be a bit more awkward, but yeah, can do. – IamPancakeMan May 29 '19 at 15:01
  • @HereticMonkey SetInterval could very useful if done quickly and only if first read throws. Would this get a bit busy for the server, though? There are probably other things for Node to be doing in this time. – IamPancakeMan May 29 '19 at 15:05
  • 1
    As long as you keep the amount of things you do in the interval small, I doubt this would tax the server much, but of course you should test to ensure that. `setInterval` allows for other code to be run between intervals. – Heretic Monkey May 29 '19 at 18:39

1 Answers1

-1

May be it will be over for this story, but you can create own fs with full control. In this case other programs will write data directly to your program. Just search by word fuse and fuse-binding

Yaroslav Gaponov
  • 1,997
  • 13
  • 12