2

I am writing a small text file (~500B) but, strangely, I get an empty file if I write using asynchronous methods such as fs.writeFile(..) (or WriteableStream's write/end method).

This works:

var scanInfo = getScanInfo( core ); // returns several lines delimited by \r\n
fs.writeFileSync( filename, scanInfo, 'ascii' );

This creates empty file and the callback function never produces any output:

var scanInfo = getScanInfo( core );
scanInfo.push('') ;
scanInfo = scanInfo.join(DOS_CRLF);
fs.writeFile( filename, scanInfo, 'ascii', function ( err ) { 
   if(err) { console.error('Failed'); console.error(err) ; }
   else { console.log('OK'); }
});

I was looking for similar posts but in the one I found the problem was something else (calling another function returning the content) but my content is a text string (verified by debugging).

The similar post: fs.writeFile() doesn't return callback

Platform> Win8.1 x64

NodeJS> x64 0.12.0

P.S. The application using the function that is actually writing the file was written in a "plain nodejs" style using callbacks but as it got more complicated I rewrote the main processing stream using Q and Q-IO. So now the processing starts like this:

(in the main module)

var qfs = require('q-io/fs') ;
...
qfs.read( configFile )
.then( doSomeConfig )
.then( function( config ) { 
    var promise = qfs.read( config.inputFile, someOptions );
    return promise ;
})
.then( processMyInputData /* (binaryData) returns {Core} */ )
.then( writeMyOutputData  /* (core)   returns {undefined} */  )
.fail( reportSomeErrors   /* (reason) returns {undefined} */ )
.done( reportFinished ) ;

The point is that in the main stream the fail function never reports any problem, either. Function reportFinished() reports that everything was OK and there is no place to throw any exception because the original snippet above, which is a function located in another module and called as part of writeMyOutputData( core ) never gets to call the callback and therefore it is not possible to do any exception throwing or any kind of error processing.

However, after reading Joseph's comment that it works for him I suspect there might be some interference between the standard fs module and q-io/fs

Community
  • 1
  • 1
user294943
  • 321
  • 1
  • 3
  • 9
  • Do you close the file? – jfriend00 May 18 '15 at 15:32
  • What is `DOS_CRLF`, just a string `\r\n`? If I simulate your code in node it seems to work all fine. Can I ask what your filename is? And also important, what platform are we talking about? Linux, Mac or Windows? – Joseph Callaars May 18 '15 at 15:52
  • What node version and what OS? – mscdex May 18 '15 at 16:01
  • Sorry, I copied the code and forgot few things: platform is Win 8.1-x64, nodejs is x64 0.12.0, DOS_CRLF = '\r\n' ; Filename is something like "C:\\DATA\\out\\cv1076_ScanInfo.csv", the text to be written is a string (getScanInfo() returns array and in both snippets it is converted to the same string by join(DOS_CRLF)). I run it in Windows during the development and the production target will be Linux. – user294943 May 19 '15 at 09:52
  • I agree that the interference is somewhere before the function being called. Can you debug through it as I think it might not come at the `writeFile` at all. – Joseph Callaars May 19 '15 at 11:59
  • Shame on me! The key point is that there is an exception that does not get caught when the files are written asynchronously. In some sense I am disappointed by node. Cause of the problem: two files are written concurrently and asynchronously (ScanInfo and another one). The other one includes on-the-run average calculation and contains a typo in variable name (causing Reference Error). – user294943 May 19 '15 at 14:04
  • Result: the exception is masked by creating a new promise in the promise chain of the top-level process, where the first promise has no fail handler. Due to the uncaught exception somewhere on the way the fs callbacks are never called and neither ever handled by higher-level promises. I rewrote the export of ScanInfo and the related file to also use q-io and added at least one fail handler in each chain, and voila, the error popped up. I think this question should be closed as it has no relevance to callbacks or fs, it is a matter of bad programming style and inappropriate use of promises. – user294943 May 19 '15 at 14:07

2 Answers2

6

OK, after careful deugging problem identified. As Joseph mentioned, not related to fs.writeFile() at all.

In my application there are in fact two file writes running "concurrently". The one listed in my question and another one, writing data progressively as it calculates some averages.

The other, progressively writing function, had a bug (misspelled variable name), causing a Reference Error to be thrown in the course of action (in between successive writes). This exception, for some reason I do not quite understand, did not appear anywhere in the chain. According to Q documentation, Promise.done() should throw any unhandled exceptions, but this was not the case.

After I added several fail() handlers in the promise chain, I was able to locate the bug and achieve reasonable behavior of the whole application.

The error is therefore related to bad programming style (not handling exceptions properly) rather than fs module. However, I can't believe that there could be such thing as unhandled exception that can get lost and never appear in the daylight. Also I can hardly believe that an exception in asynchronous operation B can affect another, non-related asynchronous operation A.

user294943
  • 321
  • 1
  • 3
  • 9
  • 1
    omg thank you! you saved many days of work. A fs callback wasn't getting called and I had no clue why I was trying to debug node next step would have been trying to debug node source when I forgot a little await in my code ^^ – Vinz243 Feb 14 '17 at 13:49
0

I had a similar issue with fs.stat The issue was that i was writing a grunt task and the task didn't know it was asynchronous so the synchronous code finished and just terminated the application before the fs.stat callback could be called.

This is probably not your issue, but it might help others.

Making a grunt task can be done like this: Wait async grunt task to finish

Community
  • 1
  • 1
Berty
  • 1,081
  • 1
  • 18
  • 49