6

I am working on a C# program that needs to use the FileSystemWatcher class so it will be notified when new files are created. As part of the initialization, the program scans the directory so it can process any files that already exist in it. This is all working fine.

However, in a discussion with another developer, we started questioning whether this will always work. Are there conditions under which the FileSystemWatcher will miss the creation of files? If so, what are these conditions?

In order to handle this scenario, we would just run the code in our initialization process that scans the directory periodically, but how likely is it for the FileSystemWatcher to miss files?

Tony Vitabile
  • 8,298
  • 15
  • 67
  • 123
  • This previous question / answer should give you a good start: http://stackoverflow.com/questions/239988/filesystemwatcher-vs-polling-to-watch-for-file-changes. In short the Win32 buffer underlying the FileSystemWatcher can overflow – Kevin Jan 08 '14 at 16:14
  • 1
    possible duplicate of [How reliable is the FileSystemWatcher in .netFramwork 4?](http://stackoverflow.com/questions/7191380/how-reliable-is-the-filesystemwatcher-in-netframwork-4) – Yuck Jan 08 '14 at 16:14

3 Answers3

14

FileSystemWatcher will not usually miss files. However:

  • since it's based on ReadDirectoryChangesW, it only detects changes to the file's directory entry, not changes to the file itself. Most changes to the file will update the directory entry, but there are a few exceptions (see this article).
  • the buffer for file change notifications has a limited size; if you don't process the events fast enough, the buffer will overflow, causing you to miss events. That's why you should not do any heavy processing in the event handler; if you can't handle the events fast enough, just add them to a queue that you process on another thread.

Other pitfalls:

  • The notifications don't arrive instantly; the delay between the actual change and the notification is usually very short, but I've seen it grow to as long as several seconds. This is not a major issue for most use cases, but depending on what you're trying to do, it could be a problem.
  • There is no event for Moved. If you move a file from a directory to another, you will receive two notifications : Deleted and Created
  • Sometimes, depending on the volume configuration, the paths in the notifications can be in the old 8.3 format (e.g. you can get SOMETH~1.TXT instead of Something.txt)
  • If you move an existing, non-empty directory into the watched directory, you will only get a notification for the directory itself, not its content. You will need to examine the content manually.
  • Changed events can occur several times for the same file; you need to handle the duplicates yourself
Thomas Levesque
  • 286,951
  • 70
  • 623
  • 758
  • Thanks for that detailed response. We already knew about the multiple Changed events and the Move pitfall. I've read about moving a non-empty directory issue earlier today. I did not know about the directory entry vs. file itself issue. Fortunately, I'm only interested in new files being added, not changes to existing files. Thanks again! – Tony Vitabile Jan 08 '14 at 18:09
2

The FileSystemWatcher is not supposed to miss any file, however, if your file processing is long, you should queue the events and treat them in a different thread.

Uri Y
  • 840
  • 5
  • 13
0

I have used fs watcher time-to-time, the biggest challenge always that the FSwatcher created event trigger when the file created (without actually final size), it means if somebody inserting in a fileshare a huge file "gigs" that will be locked until the file released by the creator app, so I can see perfect in a test environment, where limited files and data can appear. For production fswatcher combined with background worker and timer can be the solution eg FSwatcher created even should handover the task to a backgroundworker. Still i saw some cases where i need to have a hourly timer to move the remaining files, there were some weird cases when user was still using the files in couple hours. I found the perfect solution use FSwatcher along with multi threading and timer. I assume you want a windows service not a form based or web based app. For file changed event, I would not use this, if any app open a file in filestream it will execute couple of changes, but still locking the file, so you can do nothing with it, maybe you can add to list to be able to deal with it later , but as far i know in that case, if the file is edited in stream it will keep firing you the event. So my suggestion use created event set some timeout in the event to check really fast read avaiability like 5 second, if this is not happen spawn a background thread , and even the bg tread still cannot deal with the file, have a timer which keep retry on all files, which was not handled in both scenarios (like huge file and service terminated, then your background thread disposed).

Summary FSwatcher raising the event, but that not means files are accessible and ready for processing, it just monitor the filesystem and fire events you subscribed to. The events firing, but if you are not dealing with it fast enough, you may miss the next one, so use event for very fast actions, if you have lot files created, like adding them to a list, and a separate threat to do the task! If you spend too much time in an event, you might miss some, so catch the event, handover immediately the task and wait for the next one:)

Szili
  • 1