FileSystemWatcher sometimes not working

Question

I have a FileSystemWatcher set to check for new files, store the content in a database and delete the files. About 10 days ago, it started to ignore some files. We're talking about 1,500 files from a total of 50,000 files. By manually moving the files to a different directory and then moving them again to the watched directory, the files get noticed.

The InternalBufferSize is set to 32 kB to deal with large batches. It processes 300+ files at once without a problem, and reality isn't even close to that.

The program was last touched over 40 days ago, for a change unrelated to the FileSystemWatcher. It's been in production for over a year now. No spikes can be seen in the server load.

What can cause an issue like this one to suddenly appear? Is there a possibility that FileSystemWatcher is simply unreliable?

Edit I've created a test where 1,000 files are created. After running it, 3,000 entries can be found in the event log. So I believe buffer overflow is out of the question?

    private void button1_Click(object sender, EventArgs e)
    {
        fsw = new FileSystemWatcher();
        fsw.Path = @"C:\temp\fsw-test";
        fsw.IncludeSubdirectories = false;
        fsw.NotifyFilter = NotifyFilters.FileName;
        fsw.Created += new FileSystemEventHandler(fsw_Created_handler);
        fsw.EnableRaisingEvents = true;
        fsw.InternalBufferSize = 32768;
        fsw.Error += fsw_Error_handler;
    }

    private void fsw_Created_handler(object sender, FileSystemEventArgs e)
    {
        new Thread(new ParameterizedThreadStart(work)).Start(e);
    }

    private void fsw_Error_handler(object sender, ErrorEventArgs e)
    {
        EventLog.WriteEntry("few test", e.GetException().Message);
    }

    private void work(object e)
    {
        try
        {
            EventLog.WriteEntry("fsw test", "Queueing File Started");
            Thread.Sleep(10000);
            EventLog.WriteEntry("fsw test", ((FileSystemEventArgs)e).Name);
            EventLog.WriteEntry("fsw test", "Queueing File Done");
        }
        catch (Exception ex)
        {
            EventLog.WriteEntry("fsw test", "Error = " + ex.StackTrace + " *** " + ex.ToString());
        }
    }

    private void button2_Click(object sender, EventArgs e)
    {
        for (int i = 1; i <= 1000; i++)
        {
            System.IO.File.Create(@"C:\temp\fsw-test\" + i);
        }
    }

Edit 2 Stress testing the program in multiple ways and checking the code over and over again revealed no issues. So right now it's an unreproducible bug, I'll do a few changes to make it log more frequently and monitor the situation.

The only advice I can give is that some of the options to the `FileSystemWatcher` were quite cryptic. When I used it, I set up what I thought would catch the reads/writes/edits, but it wasn't catching all of them. Perhaps some new "cases" of how files are being "created" have appeared and the FSW isn't set up to catch them, but it is still set up to catch the "moves" which is why moving them from one directory to another works. — Origin, Feb 13 '12 at 15:46
@adrianm It's not being monitored, only `Created` is being monitored. I'll add it for extended logging, but I highly doubt it overflows. — user247702, Feb 13 '12 at 15:54
It is not just overflow that is reported via `Error`, (e.g. network errors). In my experience there are 2 rules to follow for a reliable watcher. 1. Don't do any work in the handler (just save the event information somewhere and signal a worker thread). 2. listen to the error event and restart/recreate the watcher after each error. — adrianm, Feb 13 '12 at 15:59
Then MSDN is incomplete. The first rule is already implemented, I'll see if the second rule is a possibility (as opposed to dropping it, see my comment to @Steve's answer). — user247702, Feb 13 '12 at 16:02

score 3 · Answer 1 · answered Feb 13 '12 at 15:47

3

The file events are not queued up. If you are processing a file, and new files are created, you will miss the events.

One way to work around this, when a new file event occurs, work on the file and before returning, check for new files. Loop until no files are left.

answered Feb 13 '12 at 15:47

Steve Wellens

20,506
2
28
69

If this is true, then there's an inherent race (even if it's small) that might be impossible to workaround - after the event handler decides that there are no new files there will always be some short period of time when file system changes can occur before control returns to whatever initiates the events. – Michael Burr Feb 13 '12 at 15:51
The handler for the `Created` event starts a new thread every time to do the actual processing. As noted in the question, I can drop 300+ files in the directory and every single one of them will be processed. Or am I misunderstanding what you mean? The workaround is a possibility indeed, but then I might as well drop the FileSystemWatcher completely and periodically poll the directory for files. – user247702 Feb 13 '12 at 15:57
You can verify it easily. In the handler for the event, put in a 10 second sleep. While that code is sleeping, create a new file and see if you get another event. I suggested one workaround that worked for us, I'm sure there are others. – Steve Wellens Feb 13 '12 at 16:03
I've tried what you said, and I actually do get another event, after the 10 second sleep is done. `private void fsw_Created_handler(object sender, FileSystemEventArgs e) { new Thread(new ParameterizedThreadStart(work)).Start(e); Thread.Sleep(10000); }` – user247702 Feb 13 '12 at 16:30
1

Yeah, but you are creating a thread and returning the watcher to active mode immediately. That is not the test I suggested. Note: There is an internal buffer you can set. (Why didn't I see this years ago?) http://msdn.microsoft.com/en-us/library/system.io.filesystemwatcher.internalbuffersize%28v=vs.80%29.aspx. But it confirms what I said: "Note that an instance of the FileSystemWatcher class does not raise an Error event when an event is missed or when the buffer size is exceeded, due to dependencies with the Windows operating system. " – Steve Wellens Feb 13 '12 at 16:39
@Steve that's the real scenario. But does the 10 second sleep not keep the handler busy? – user247702 Feb 13 '12 at 16:42
The sleep would have to be in the thread of the event handler, not a new thread (Note: I edited my previous response). – Steve Wellens Feb 13 '12 at 16:44
I've added a test to the question. It uses the same buffer size as the program and handles a much higher amount of files at once than we actually need to handle. – user247702 Feb 13 '12 at 16:46
@SteveWellens How would you check for new files before returning? – Frying Pan Apr 21 '21 at 22:24
GetFiles( @"C:\temp\fsw-test"); This assumes after you process a file you delete it or move it out of the directory. – Steve Wellens Apr 22 '21 at 04:40

score -1 · Accepted Answer · answered Mar 11 '12 at 12:04

-1

The problem turned out to be a Queue corruption, caused by multi-threading.

answered Mar 11 '12 at 12:04

user247702

23,641
15
110
157

FileSystemWatcher sometimes not working

2 Answers2