1

The following "write file", "check hash" in intermediate cases ( only sometimes) fails in an Electron 8.2.1 application. It will display that it has calculated d41d8cd98f00b204e9800998ecf8427e which is the hash over an empty string. However when i look in the folder, the file exists. So my assumption is that somehow sometimes that fs.writeFile under higher load still is not ready writing or something the file returns an empty string instead of the contents. And note that in 99% of the cases the application runs correctly. Only in some cases (and we think very high load) this fails.

I read node - fs.writeFile creates a blank file which comes closer but it does not provide a reason or "why the hell" nor because of this you need to do that.

fs.writeFile(cPath, body, 'utf-8', (err) => {
    if (err) {
      errors.handleErrorLocal(err);
      reject();
      return;
    }
    log.info(cFile + ' C HASH is:' + hash + ' HASH calculated:' + md5File.sync(cPath))
    if (hash && md5File.sync(cPath) !== hash.toLowerCase()) {
       log.warn('C failed to download. Wrong Hash. ', cFile)
    }
    resolve();
});

The answer on node.js readfile woes from panu-logic seems to correspond with this experience (bottom answer) but also does not provide any reason why other than "magical" in his case however he tries to read while being written. "logically" this is not the case but the assumption is that this is for some unknown reason the case.

I am aware that i could rewrite to stream or await but that requiress me to change it , push it out, waiting for some time for bugs then retry. I would rather know before all of this what the reason is and just know for sure that the fix for this works rather than stretching this over weeks or months.

edelwater
  • 2,650
  • 8
  • 39
  • 67
  • (1) Is there a way you could make a complete program which will (hopefully) also show the faulty behavior? It should be possible to run it a large number of times to evoke the rare problem. – snwflk Sep 16 '20 at 16:41
  • (2) Just to be sure, it is the call to `md5File.sync` that gives `md5("")`? Please include the import – but that should already be covered by (1). – snwflk Sep 16 '20 at 16:41
  • (3) How large is the file typically that we are talking about? Things are different between a few hundred bytes and a 500 GB file. – snwflk Sep 16 '20 at 16:48
  • hello snwflk, thank you for your comments! ( 1 ) yes this is a complete application in electron which is running on a lot of machines 24/7 so therefore , since on average it goes ok, it seems to be when the load is high on a machine, different types ( 2 ) yes , the log line i included shows that the MD5 operates against an empty string , the import for the MD5 is simply const md5File = require('md5-file'); or in package.json "md5-file": "^4.0.0", – edelwater Sep 16 '20 at 22:55
  • ( 3 ) no sometimes just a few kb, could even be a very small html file. And most often it happens in batches, so a range of those then fail while on a next run they run ok. but it causes problems in that duration of time. || if I read behind the lines on what you are writing you are suspecting that md5File.sync versus Md5File.then would be the culprit ? – edelwater Sep 16 '20 at 22:55
  • Sorry, I wasn't clear on (1). Please write a small new program that other people can run in order to reproduce your problem. Post the code here by editing your question. Here are actually helpful tips on how to do it: [example] – snwflk Sep 17 '20 at 17:41

0 Answers0