StreamWriter even with AutoFlush true leaves lines half written

Question

I have a StreamWriter with an AutoFlush = true property. However, I still see the file only partially written when I randomly open it. I'm writing a file that needs to be fully written (JSON) or not during any given time.

        var sw = new StreamWriter("C:\file.txt", true /* append */, Encoding.ASCII) { AutoFlush = true };
        sw.WriteLine("....");

        // long running (think like a logging application) -- 1000s of seconds

        sw.Close();

In between the sw.WriteLine() call and sw.Close() I want to open the file, and always have it be in the "correct data format", i.e. my line should be complete.

Current Idea:

Increase the internal buffer of FileStream (and/or StreamWriter) to let's say 128KB. Then every 128KB-1, call .Flush() on the FileStream object. This leads me to my next question, when I do call Flush(), should I right before calling it get the Stream.Position and do a File.Lock(Position, 128KB-1)? Or does Flush take care of that?

Basically I don't want the reader to be able to read the contents in between Flush(), because it'll maybe partially broken.

If you use NTFS and don't care about Windows XP, you can try to use transactions - [How do I open a Windows 7 transacted file in C#](http://stackoverflow.com/questions/3267595/how-do-i-open-a-windows-7-transacted-file-in-c-sharp). Or you can create a copy of file, write content there and then overwrite the original one by move operation. — Ulugbek Umirov, Mar 29 '14 at 20:15

score 2 · Answer 1 · answered Mar 29 '14 at 20:05

2

using (StreamWriter sw = new StreamWriter("FILEPATH"))
{
  sw.WriteLine("contents");
  // if you open the file now, you may see partially written lines
  // since the sw is still working on it.
}

// access the file now, since the stream writer has been properly closed and disposed.

answered Mar 29 '14 at 20:05

Raja Nadar

9,409
2
32
41

That's not what the documention says – Teejay Nov 06 '19 at 11:14
(if you use AutoFlush = true) – Teejay Nov 06 '19 at 14:37

score 0 · Answer 2 · answered Dec 10 '18 at 09:57

0

If you need a "log-like" file which is never half-written, the way to go is not keeping it open.

Every time, you want to write your file, you should instantiate a new FileWriter, which will flush the file contents upon releasing the file like this:

private void LogLikeWrite(string filePath, string contents)
{
  using (StreamWriter streamWriter = new StreamWriter(filePath, true)) // the true will make you append to the file instead of overwriting its contents
  {
    streamWriter.Write(contents);
  }
}

This way your write operations will be flushed immediately.

answered Dec 10 '18 at 09:57

mg30rg

1,311
13
24

1

I find this hard to believe. If it's true (closing is the only way to flush) then the existence of `AutoFlush` and `Flush()` makes no sense. – Aug 05 '19 at 12:39
@WumpusQ.Wumbley If you check out [MSDN on the topic](https://learn.microsoft.com/en-us/dotnet/api/system.io.streamwriter.flush?view=netframework-4.8#remarks), you can see that in certain cases calling `.Flush()` or using `.AutoFlush` won't result in the expected behavior. this is why applying the logging method _which is basically industry standard for more than four decades_ is recommended. – mg30rg Aug 05 '19 at 13:11
For only 2.5 decades, I've used C, and never had to worry that `fflush` didn't do what it was supposed to do (also there was line-buffering mode as an option), and constantly closing and reopening a file to make sure it was flushed would have been laughed at. Only in Microsoft-world could such an ugly workaround be "industry standard" – Aug 05 '19 at 13:45
@WumpusQ.Wumbley are you aware of the fact, that line-buffering actually closes and reopens the filestream? – mg30rg Aug 05 '19 at 15:05
If it's *log-like* why not just `File.AppendAllText(filePath, contents)`? Is there any advantage of your function over the built-in? – Alb Apr 03 '20 at 00:41
@Alb in this special case, the advantage of my solution, is that it utilises a filestream, and the OP was asking about a filestream-based solution. Maybe they are learning to code, and simply aren't aware of IO.File. Maybe they wish to maintain uniformity with the rest of the codebase. Maybe it is just aesthetics. Who am I to question it without knowing the details? – mg30rg Apr 03 '20 at 12:22
Although implementation can change `AppendAllText()` does utilize a filestream in the same way. We have `WriteAllText()` as equivalent for `append: false` and so on. After all I partially agree, `Log(...);` is more meaningful. Thanks for your opinion. – Alb Apr 03 '20 at 12:59

score 0 · Answer 3 · answered Dec 14 '18 at 23:57

If you are sharing the file between processes, your going to have a race condition unless you produce a locking mechanism of some kind. See https://stackoverflow.com/a/29127380/892327. This does require that you are able to modify both processes.

An alternative is to have process A wait for a file at a specified location. Process B writes to a intermediate file and once B has flushed, the file is copied to the location process A is expecting a file to be so that it can consume the file.

StreamWriter even with AutoFlush true leaves lines half written

3 Answers3