Checking for files modifications

Question

What I'm doing

I'm working on a webservice which is copying files from one location to another. Files are being updated(the size should be increased every 3 seconds since there is text added).

1st option:

I'm checking every 10 seconds if any of the files is modified(they are being modified every 5 seconds cca) so I can copy(and overwrite) them to the final destination. Atm I'm using a code which is comparing the last edit time of the file with actual time - some amount of time(1 minute atm).

DateTime lastEditTime = new DateTime();
lastEditTime = File.GetLastWriteTime(myFile);

if (lastEditTime > DateTime.Now.AddMinutes(-1)) 
{
     File.Copy(myFile, newFileName, true);
}

But I think this is kinda bad approach since there might be some time space or something and I won't get some changes.

2nd option

I could check the file sizes(using the FileInfo.Length property probably) of each file in the source directory and compare them to the ones in final destination. This should be ok too since the file sizes should only grow since the text is added only.

3rd option

I read a lot people recommend the FileSystemWatcher but I don't want to miss some changes which might happen - at least I read that at other SO questions(see https://stackoverflow.com/a/240008/2296407).

What is my question?

What is the best option to know if any file was changed(if the file in source is different from file in final destination) in last x mins or seconds cause I don't want to copy everything since there might be a lot of files. By being best option I mean: is it faster to compare each files sizes or compare the File.GetLastWriteTime(myFile) with actual time - some time. In the second case ther is also question: How big the time span should be? If I put a big time span I will probably copy more files than I actually need but if I put it low I might miss some changes.

If you have some better options feel free to share them with me!

Answer 1

Although you already mentioned it in your option 3, I still think should give it a try with the FileSystemWatcher Class. As far as I understood you, you have not yet done so, right?

Although it is true that the watcher may lose some event in the default configuration, you can still make it work reliably if you do some tweaking.

Have a look at the "Remarks" section in the documentation (highlights by me):

The Windows operating system notifies your component of file changes in a buffer created by the FileSystemWatcher. If there are many changes in a short time, the buffer can overflow. This causes the component to lose track of changes in the directory, and it will only provide blanket notification. Increasing the size of the buffer with the InternalBufferSize property is expensive, as it comes from non-paged memory that cannot be swapped out to disk, so keep the buffer as small yet large enough to not miss any file change events. To avoid a buffer overflow, use the NotifyFilter and IncludeSubdirectories properties so you can filter out unwanted change notifications.

Things you can do to make it reliably work:

Note that a FileSystemWatcher may miss an event when the buffer size is exceeded. To avoid missing events, follow these guidelines:

• Increase the buffer size by setting the InternalBufferSize property.
• Avoid watching files with long file names, because a long file name contributes to filling up the buffer. Consider renaming these files using shorter names.
• Keep your event handling code as short as possible.

For example user Nomix says he raised the buffer size (property InternalBufferSize) up to 16 MB and has never had a problem with the FileSystemWatcher class (SO post is here.) And I can confirm this with a project in my company that works fine for years, too, since we found out about the buffer.

Initialization of the object might look like this for example:

private void InitWatcher()
{
    // Create a new FileSystemWatcher and set its properties.
    FileSystemWatcher watcher = new FileSystemWatcher();
    watcher.Path = "Your path to watch";

    // You only want to watch a single folder
    watcher.IncludeSubdirectories = false;

    // You mentioned both LastWrite and Size
    // You can combine them or just watch for only a specific property
    // Simply configure it to your needs
    watcher.NotifyFilter = NotifyFilters.LastWrite | NotifyFilters.Size

    // Only watch text files.
    watcher.Filter = "*.txt";

    // Add event handlers, omit those you are not interested in
    watcher.Changed += new FileSystemEventHandler(OnChanged);

    // Begin watching.
    watcher.EnableRaisingEvents = true;
}

You can then subscribe to those events that are of interest for you, like the Changed event and react to it as easy as:

private static void OnChanged(object source, FileSystemEventArgs e)
{
   File.Copy(e.FullPath, newFileName, true);
}