System.IO.File.Delete() / System.IO.File.Move() sometimes does not work

Question

A Winforms program needs to save some run time information to an XML file. The file can sometimes be a couple of hundred kilobytes in size. During beta testing we found some users would not hesitate to terminate processes seemingly at random and occasionally causing the file to be half written and therefore corrupted.

As such, we changed the algorithm to save to a temp file and then to delete the real file and do a move.

Our code currently looks like this..

private void Save()
{
    XmlTextWriter streamWriter = null;
    try
    {
        streamWriter = new XmlTextWriter(xmlTempFilePath, System.Text.Encoding.UTF8);

        XmlSerializer xmlSerializer = new XmlSerializer(typeof(MyCollection));

        xmlSerializer.Serialize(streamWriter, myCollection);

        if (streamWriter != null)
            streamWriter.Close();

        // Delete the original file
        System.IO.File.Delete(xmlFilePath);

        // Do a move over the top of the original file 
        System.IO.File.Move(xmlTempFilePath, xmlFilePath);
    }
    catch (System.Exception ex)
    {
        throw new InvalidOperationException("Could not save the xml file.", ex);
    }
    finally
    {
        if (streamWriter != null)
            streamWriter.Close();
    }
}

This works in the lab and in production almost all of the time. The program is running on 12 computers and this code is called on average once every 5 min. About once or twice a day we get this exception:

System.InvalidOperationException: 
Could not save the xml file. 
---> System.IO.IOException: Cannot create a file when that file already exists.
at System.IO.__Error.WinIOError(Int32 errorCode, String maybeFullPath)
at System.IO.__Error.WinIOError()
at System.IO.File.Move(String sourceFileName, String destFileName)
at MyApp.MyNamespace.InternalSave()

It is as if the Delete is not actually issued to the hard drive before the Move is issued.

This is happening on Win7 machines.

A couple of questions: Is there some concept of a Flush() one can do for the entire disk operating system? Is this a bug with my code, .net, the OS or something else? Should I be putting in some Thread.Sleep(x)? Maybe I should do a File.Copy(src, dest, true)? Should I write the following code? (But it looks pretty silly.)

while (System.IO.File.Exists(xmlFilePath))
{
    System.IO.File.Delete(xmlFilePath);
}

// Do a move over the top of the main file 
bool done = false;
while (!done)
{
    try
    {
        System.IO.File.Move(xmlTempFilePath, xmlFilePath);
        done = true;
    }
    catch (System.IO.IOException)
    {
        // let it loop
    }
}

Has anyone seen this before?

As an aside you should be catching the specific exceptions rather than the base `Exception`. — ChrisF, Jun 24 '11 at 11:17
Do you run a virus-scanner on those folders? And who reads those files and when? Related: http://stackoverflow.com/q/6350224/60761 — H H, Jun 24 '11 at 11:27
@ChrisF: This is one of those situations where catching `Exception` is perfectly fine. He's wrapping exceptions, not swallowing them. — Gabe, Jun 24 '11 at 11:44
There are virus scanners running on these computers. They would be targeting this folder for sure. Irrespective, a delete is a delete. The code still needs to work reliably. I can't suggest that people disable their virus scanner. — DJA, Jun 24 '11 at 12:04

Hans Passant · Answer 1 · 2011-06-24T13:05:32.920

You can never assume that you can delete a file and remove it on a multi-user multi-tasking operating system. Short from another app or the user herself having an interest in the file, you've also got services running that are interested in files. A virus scanner and a search indexer are classic trouble makers.

Such programs open a file and try to minimize the impact that has by specifying delete share access. That's available in .NET as well, it is the FileShare.Delete option. With that option in place, Windows allows a process to delete the file, even though it is opened. It gets internally marked as "delete pending". The file does not actually get removed from the file system, it is still there after the File.Delete call. Anybody that tries to open the file after that gets an access denied error. The file doesn't actually get removed until the last handle to the file object gets closed.

You can probably see where this is heading, this explains why File.Delete succeeded but File.Move failed. What you need to do is File.Move the file first so it has a different name. Then rename the new file, then delete the original. Very first thing you do is delete a possible stray copy with the renamed name, it might have been left behind by a power failure.

Summarizing:

Create file.new
Delete file.tmp
Rename file.xml to file.tmp
Rename file.new to file.xml
Delete file.tmp

Failure of step 5 is not critical.

Good answer. Is there any reference for the assertion of FileShare.Delete's behavior? — Johannes Rudolph, Nov 30 '12 at 08:30

Bek Raupov · Accepted Answer · 2011-06-24T11:31:37.590

3

how about using Move.Copy with overwrite set to true so that it overwrites your app state and then you can delete your temp state ?

You can also attach to App_Exit event and try to perform clean shut down?

edited Jun 24 '11 at 11:31

answered Jun 24 '11 at 11:19

Bek Raupov

3,782
3
24
42

And how does that answer the question. – H H Jun 24 '11 at 11:25
Copy with overwrite might work where a delete then move doesn't considering the error is "file already exists". It could be worth a try. – Phil Gan Jun 24 '11 at 11:41

score 1 · Answer 3 · answered Jun 24 '11 at 11:46

If multiple threads in this application can call Save, or if multiple instances of the program are attempting to update the same file (on e.g. a network share), you can get a race condition such that the file doesn't exist when both threads/processes attempt the delete, then one succeeds in performing its Move (or the Move is in progress), when the second attempts to use the same filename and that second Move will fail.

As anvarbek raupov says, you can use File.Copy(String, String, Boolean) to allow overwriting to occur (so no longer need the delete), but this means that the last updater wins - you need to consider if this is what you want (especially in multi-threaded scenarios, where the last updater may, if you're unlucky, have been working with older state).

Safer would be to force each thread/process to use separate files, or implement some form of file system based locking (e.g. create another file to announce "I'm working on this file", update the main file, then delete this lock file).

I have not mentioned it in the post, but the code has been very carefully protected via a system wide mutex. Concurrency is not an issue. — DJA, Jun 24 '11 at 12:00

score 0 · Answer 4 · answered Jun 24 '11 at 13:08

As Hans said, a workaround is to move the file first and THEN delete it.

That said with Transactional NTFS' delete, I haven't been able to reproduce the error described. Check out github.com/haf/Castle.Transactions and the corresponding nuget packages... 2.5 is well tested and documented for file transactions.

When testing with non transacted file systems, the unit-test code for this project's unit tests always does moves before deletes.

3.0 is currently in pre-alpha state but will integrate regular transactions with file io with transactional files to a much higher level.

System.IO.File.Delete() / System.IO.File.Move() sometimes does not work

4 Answers4

Linked