IOException happening despite thread synchronization C#

Question

I have setup 2 event handlers both declared within the same class (let's call it WrapperClass): one to save files to a folder and another to send those files to a web api. In the application main thread, I called the two methods at application start-up:

// save file to folder:
NewFileEventHandler();
// send file to api:
FileTransporter();

NewFileEventHandler is defined as below:

public void NewFileEventHandler()
{
    SomeEventClass.NewFileEvent +=
         new FileEventHandler(SaveFileToFolder);
}

private void SaveFileToFolder(File file)
{
    FileHelper.SaveFileToFolder(file);
}

FileTransporter is defined as below, which is where I'm getting the issue with:

public void FileTransporter()
{
     FileSystemWatcher newFileWatcher = new FileSystemWatcher();
     newFileWatcher.Path = ConfigurationHelper.applicationRootDirectory;
     newFileWatcher.Filter = "*.txt";
     newFileWatcher.Created +=
     new FileSystemEventHandler(TransportFile);
     newFileWatcher.EnableRaisingEvents = true;
}

And the `TransportFile()` is given below:

private void TransportFile(object source, FileSystemEventArgs e)
{
    lock (_fileTransportLock)
    {
         string[] files = new string[] { };
         files = Directory.GetFiles(ConfigurationHelper.applicationRootDirectory, "*.txt", SearchOption.TopDirectoryOnly);
         Parallel.ForEach(files, (currentFile) =>
         {
             bool isHttpTransferSuccess = false;

             isHttpTransferSuccess = FileHelper.SendFileToApi(userid, currentFile);
             if (isHttpTransferSuccess)
             {
                 File.Delete(currentFile);
             }
        });
     }
}

However, the line:

throws the exception:

System.IO.IOException: The process cannot access the file 'C:\Users\myapp\file.txt' because it is being used by another process.
   at System.IO.__Error.WinIOError(Int32 errorCode, String maybeFullPath)
   at System.IO.FileStream.Init(String path, FileMode mode, FileAccess access, Int32 rights, Boolean useRights, FileShare share, Int32 bufferSize, FileOptions options, SECURITY_ATTRIBUTES secAttrs, String msgPath, Boolean bFromProxy, Boolean useLongPath, Boolean checkHost)
   at System.IO.FileStream..ctor(String path, FileMode mode, FileAccess access, FileShare share)
   at System.IO.File.Open(String path, FileMode mode)
   at FileHelper.SendFileToApi(String userId, String fileLocation)

What I don't understand is, because of the lock the only possible two processes that can be using this file are the thread that is saving the file and the thread that is trying to send the file to the api. However, my understanding of the FileSystemWatcher.Created event is that it triggers when the creation of the file is complete. Which means, the thread that is saving the file should not be using the file by the time TransportFile() method try to open the file to send it to api.

Sometimes there are more than one file in the folder (due to missed out emails in the past). The IOException is only thrown for the file that was just saved to folder (in other words, the file that raised the FileSystemWatcher.Created event. The other files in the folder get cleared as expected. Can anyone help please? Thanks.

Answer 1

There are a few things that you are missing here:

1. The event that you are hooking is FileCreated. This event is fired (probably unsurprisingly) when the file is created by some other process, not when that other process has finished writing the file. What's happening here is that your process is getting notified while the other process is still writing the file, and has an exclusive lock on it. From the documentation:

The OnCreated event is raised as soon as a file is created. If a file is being copied or transferred into a watched directory, the OnCreated event will be raised immediately, followed by one or more OnChanged events.

2. After the file is created, you loop over all of the files in the directory (not just the file that was just created) and transport all of them. If multiple files are created, the first call to the event will attempt to access any files in the directory (in parallel, in fact), and so even if 1) wasn't an issue, you could have a collision here with another file being written to while processing the event for the first.

The right thing to do here is to loop until you are able to read the file, as specified in the second answer here: Wait for file to be freed by process