ReplaceFile alternative when application keeps file locked

Question

Editor FooEdit (let's call it) uses ReplaceFile() when saving to ensure that the save operation is effectively atomic, and that if anything goes wrong then the original file on disc is preserved. (The other important benefit of ReplaceFile() is continuity of file identity - creation date and other metadata.)

FooEdit also keeps open a handle to the file with a sharing mode of just FILE_SHARE_READ, so that other processes can open the file but can't write to it while it while FooEdit has it open for writing.

"Obviously", this handle has to be closed briefly while the ReplaceFile operation takes place, and this allows a race in which another process can potentially open the file with write access before FooEdit re-establishes it's FILE_SHARE_READ lock handle.

(If FooEdit doesn't close its FILE_SHARE_READ handle before calling ReplaceFile(), then ReplaceFile() fails with a sharing violation.)

I'd like to know what is the simplest way to resolve this race. The options seem to be either to find another way to lock the file that is compatible with ReplaceFile() (I don't see how this is possible) or to replicate all the behaviour of ReplaceFile(), but using an existing file handle to access the destination file rather than a path. I'm a bit stuck on how all of the operations of ReplaceFile() could be carried out atomically from user code (and reimplementing ReplaceFile() seems a bad idea anyway).

This must be a common problem, so probably there's an obvious solution that I've missed.

(This question seems related but has no answer: Transactionally write a file change on Windows.)

---

Here's a minimal verifiable example showing what I am trying to achieve (updated 13:18 30/9/2015 UTC). You must supply three file names as command line arguments, all on the same volume. The first must already exist.

I always get a sharing violation from ReplaceFile().

#include <Windows.h>
#include <stdio.h>
#include <assert.h>
int main(int argc, char *argv[])
{
  HANDLE lock;
  HANDLE temp;
  DWORD  bytes;

  if (argc != 4)
  {
    puts("First argument is the project file. Second argument is the temporary file.");
    puts("The third argument is the backup file.");
  }

  /* Open and lock the project file to make sure no one else can modify it */
  lock = CreateFile(argv[1], GENERIC_READ | GENERIC_WRITE, FILE_SHARE_READ | FILE_SHARE_DELETE, NULL, OPEN_EXISTING, 0, 0);
  assert(lock != INVALID_HANDLE_VALUE);

  /* Save to the temporary file. */
  temp = CreateFile(argv[2], GENERIC_READ | GENERIC_WRITE, FILE_SHARE_READ | FILE_SHARE_DELETE, NULL, CREATE_ALWAYS, 0, 0);
  assert(temp != INVALID_HANDLE_VALUE);
  WriteFile(temp, "test", 4, &bytes, NULL);
  /* Keep temp open so that another process can't modify the file. */

  if (!ReplaceFile(argv[1], argv[2], argv[3], 0, NULL, NULL))
  {
    if (GetLastError() == ERROR_SHARING_VIOLATION)
      puts("Sharing violation as I expected");
    else
      puts("Something went wrong");
  }
  else
    puts("ReplaceFile worked - not what I expected");

  /* If it worked the file referenced by temp would now be called argv[1]. */
  CloseHandle(lock);
  lock = temp;

  return EXIT_SUCCESS;
}

---

Thanks to Hans Passant, who provided some valuable clarifying thoughts in an answer now deleted. Here's what I discovered while following up his suggestions:

It seems ReplaceFile() allows lpReplacedFileName to be open FILE_SHARE_READ | FILE_SHARE_DELETE, but lpReplacementFileName can't be. (And this behaviour doesn't seem to depend on whether lpBackupFileName is supplied.) So it's perfectly possible to replace a file that another process has open even if that other process doesn't allow FILE_SHARE_WRITE, which was Hans' point.

But FooEdit is trying to ensure no other process can open the file with GENERIC_WRITE in the first place. To ensure in FooEdit that there's no race where another process can open the replacement file with GENERIC_WRITE, it seems that FooEdit has to keep hold continuously of a FILE_SHARE_READ | FILE_SHARE_DELETE handle to lpReplacementFileName, which then precludes use of ReplaceFile().

Answer 1

Actually I think there might be a solution that doesn't involve transactions (although transactions are still available as far as I know). I haven't tried it myself, but I think on NTFS it should be possible to create a new file stream (use a long random name to ensure there are no collisions), write your data, and then rename that stream to the stream you actually wanted to write to.

FILE_RENAME_INFORMATION suggests this should be possible, since it talks about renaming data streams.

However, this would only work on NTFS. For other file systems I don't think you have a choice.

Answer 2

I'd like to know what is the simplest way to resolve this race.

There is no simple way to resolve this race. It's an inherent part of the file system which is not transactional. MS introduced a transactional file API with Vista but now strongly advise developers not to use it as it may be removed in a future release.

I have had some experience with ReplaceFile but I think it caused more trouble than it was worth. My recollection was that whilst meta data was preserved, a new file was created. A consequence of this was very annoying behaviour for files saved on the desktop. Because such files have their position preserved, creating a new file resulted in the default position being used. So you'd save a file, you'd drag it to the place on the desktop where you wanted to keep it, and then when you saved the file again, it moved back to the default position.