5

My team requires a bulletproof way to save a file (less than 100kb) on Windows 10 IOT.

The file cannot be corrupted but it's OK to loose the most recent version if save failed because of power off etc.

Since the File IO has changed significantly (no more File.Replace) we are not sure how to achieve it.

We can see that: 

var file = await folder.CreateFileAsync(fileName, CreationCollisionOption.OpenIfExists);
await Windows.Storage.FileIO.WriteTextAsync(file, data);

is reliably unreliable (it repeatedly broke when stopping debugging, or reset the device.) and we are ending up with a corrupted file (full of zeroes) and and a .tmp file next to it. We can recover this .tmp file I'm not confident that we should base our solution on undocumented behaviour.

One way we want to try is:

var tmpfile = await folder.CreateFileAsync(fileName+".tmp",
                               CreationCollisionOption.ReplaceExisting);
await Windows.Storage.FileIO.WriteTextAsync(tmpfile, data);

var file = await folder.CreateFileAsync(fileName, CreationCollisionOption.OpenIfExists);

// can this end up with a corrupt or missing file?
await tmpfile.MoveAndReplaceAsync(file);

In summary, is there a safe way to save some text to a file that will never corrupt the file?

Anthony Horne
  • 2,522
  • 2
  • 29
  • 51
tymtam
  • 31,798
  • 8
  • 86
  • 126
  • Please let us know if my answer was acceptable, and please mark my answer as the answer if it helped you. – hansmbakker Aug 31 '17 at 10:27
  • We have a similar issue appending to a log file. Creating new files seems to be reliable, writing over existing files is as you say "reliably unreliable". Your new changes will be saved in the expected filename, but in a write that fails a TMP file is created that has the previous content. – Chris Schaller Nov 27 '19 at 22:45

2 Answers2

1

Not sure if there's a best practice for this, but if needed to come up with something myself:

I would do something like calculating a checksum and save that along with the file.

When saving the next time, don't overwrite it but save it next to the previous one (which should be "known good"), and delete the previous one only after verifying that the new save completed successfully (together with the checksum)

Also I would assume that a rename operation should not corrupt the file, but I haven't researched that

hansmbakker
  • 1,108
  • 14
  • 29
0

This article has a good explanation: Best practices for writing to files on the underlying processes involved with writing to files in UWP.

The following common issues are highlighted:

  • A file is partially written.
  • The app receives an exception when calling one of the methods.
  • The operations leave behind .TMP files with a file name similar to the target file name.

What is not easily deduced in discussion about the trade off with convenience-vs-control is that while create or edit operations are more prone to failure, because they do a lot of things, renaming operations are a lot more fault tolerant if they are not physically writing bits around the filesystem.

You suggestion of creating a temp file first, is on the right track and may serve you well, but using MoveAndReplaceAsync means that you are still susceptible to these known issues if the destination file already exists.

UWP will use a transactional pattern with the file system and may create various backup copies of the source and the destination files.

You can take control of the final element by deleting the original file before calling MoveAndReplaceAsync, or you could simply use RenameAsync if your temp file is in the same folder, these have less components which should reduce the area for failure.

@hansmbakker has an answer along these lines, how you identify that the file write was successful is up to you, but by isolating the heavy write operation and verifying it before overwriting your original is a good idea if you need it to be bulletproof.

About Failure

I have observed the .TMP files a lot, when using the Append variants of FileIO writing, the .TMP files have the content of the original file before Append, but the actual file does not always have all of the original client, sometimes its a mix of old and new content, and sometimes the

In my experience, UWP file writes are very reliable when your entire call structure to the write operation is asynchronous and correctly awaits the pipeline. AND you take steps to ensure that only one process is trying to access the same file at any point in time.

When you try to manipulate files from a synchronous context we can start to see the "unreliable" nature you have identified, this happens a lot in code that is being transitioned from the old synchronous operations to the newer Async variants of FileIO operations.

Make sure the code calling your write method is non-blocking and correctly awaits, this will allow you to catch any exceptions that might be raised

it is common for us traditionally synchronous minded developers to try to use a lock(){} pattern to ensure single access to the file, but you cannot easily await inside a lock and attempts to do so often become the source of UWP file write issues.

If your code has a locking mechanism to ensure singleton access to the file, have a read over these articles for a different approach, they're old but a good resource that covers the transition for a traditional synchronous C# developer into async and parallel development.

Other times we encounter a synchronous constraint are when an Event or Timer or Dispose context are the trigger for writing to the file in the first place. There are different techniques to involve there, please post another question that covers that scenario specifically if you think it might be contributing to your issues. :)

Chris Schaller
  • 13,704
  • 3
  • 43
  • 81