Out-of-mem-Exception in c# GUI when using FileSystemWatcher - multithreading

Question

I build a windows-forms-app where I (try to) do extensive calculations on images whenever they are created in a specific directory which I watch using the FileSystemWatcher.

private void OnNewFileInDir(object source, FileSystemEventArgs evtArgs) 
{  
  //Load the actual image:
  imageFilepath = evtArgs.FullPath;  //imageFilepath is a private class string var
  Image currentImage = Image.FromFile(imageFilepath);

  //Display the image in the picture box:
  UpdatePictureBox(currentImage);     //Method to update the GUI with invoking for the UI thread

  //Extensive Calculation on the images
  Image currentResultImage = DoExtensiveWork(currentImage);

  // Put the current result in the picture box
  UpdatePictureBox(currentResultImage );

  //dispose the current/temporary image
  currentImage.Dispose();
}

The event is fired correctly when pasting a new file into the directory. But I get a "System.OutOfMemoryException" on the line

Image currentImage = Image.FromFile(imageFilepath);

When I put exactly this code (using the same filepath) in a button event (so not using the FileSystemWatcher) everything works fine. So I thought there is some issue regarding the thread since the extensive calculation is then called by the FileSystemWatcher-Thread not by the UI thread.

I tried things like:

//TRY 1: By executing a button click method containg the code
pb_Calculate_Click(this, new EventArgs());    //This does not work eigther --> seems to be a problem with "Who is calling the method"

//TRY 2: Open a new dedicated thread for doing the work of the HistoCAD calculations
Thread newThread_OnNewFile = new Thread(autoCalcAndDisplay);
newThread_OnNewFile.Start();


//TRY 3: Use a background worker as a more safe threading method(?)
using (BackgroundWorker bw = new BackgroundWorker())
{
   bw.DoWork += new DoWorkEventHandler(bw_DoWork);
   if (bw.IsBusy == false)
   {
      bw.RunWorkerAsync();
   }
}

Unfortunalty none of them worked reliable. 1st not at all. 2nd works only from time to time and 3rd one as well.

Do some of you know whats going on there? What can I do to make it work correctly? Thanks!

EDIT: Thanks for the comments: I also tried to call GC.collect() on every event and tried to include using() and dispose() wherever I can. When I'm doing the process manually (with buttons) it works even when processing a lot of files one after another. But when done with the eventhandler I sometimes get the outOfMem-Exception even on the very first file I copy in the folder. File is always the same BMP with 32MB. This is the memory usage for processing one image:

EDIT 2: I created a minimal example (GUI with one picture Box and one Checkbox in buttonstyle). It turns out that the same thing is happening. The OutOfMemException occured at the same line (Image...). Especially for large BMPs the exception occours nearly always:

using System;
using System.Collections.Generic;
using System.ComponentModel;
using System.Data;
using System.Drawing;
using System.IO;
using System.Linq;
using System.Text;
using System.Threading.Tasks;
using System.Windows.Forms;

namespace MinimalExampleTesting
{
    public partial class Form1 : Form
    {
        private string imageFilepath;
        private string autoModePath = @"C:\Users\Tim\Desktop\bmpordner";

        //Define a filesystem watcher object
        private FileSystemWatcher watcher;

        public Form1()
        {
            InitializeComponent();


            /*** Creating as FileSystemEventArgs watcher in order to monitor a specific folder ***/
            watcher = new FileSystemWatcher();
            Console.WriteLine(watcher.Path);
            // set the path if already exists, otherwise we have to wait for it to be set
            if (autoModePath != null)
                watcher.Path = autoModePath;
            // Watch for changes in LastAccess and LastWrite times and renaming of files or directories.
            watcher.NotifyFilter = NotifyFilters.LastAccess | NotifyFilters.LastWrite
               | NotifyFilters.FileName | NotifyFilters.DirectoryName;
            // Only watch for BMP files.
            watcher.Filter = "*.bmp";
            // Add event handler. Only on created, not for renamed, changed or something
            // Get into the list of the watcher. Watcher fires event and "OnNewFileCreatedInDir" will be called
            watcher.Created += new FileSystemEventHandler(OnNewFileInDir);


        }

        private void tb_AutoMode_CheckedChanged(object sender, EventArgs e)
        {

            //First of all test if the auto mode path is set and correctly exists currently:
            if (!Directory.Exists(autoModePath) || autoModePath == null)
            {
                MessageBox.Show("Check if Auto Mode path is correctly set and if path exists",
                    "Error: Auto Mode Path not found");
                return;
            }

            // Begin watching if the AutoModePath was at least set
            if (autoModePath != null)
            {
                watcher.EnableRaisingEvents = tb_AutoMode.Checked;  //Since we have a toogle butten, we can use the 'checked' state to enable or disable the automode
            }

        }


        private void OnNewFileInDir(object source, FileSystemEventArgs evtArgs)
        {
            Console.WriteLine("New file in detected: " + evtArgs.FullPath);

            //Force a garbage collection on every new event to free memory and also compact mem by removing fragmentation.
            GC.Collect();

            //Set the current filepath in the class with path of the file added to the folder:
            imageFilepath = evtArgs.FullPath;

            //Load the actual image:
            Image currentImage = Image.FromFile(imageFilepath);

            UpdatePictureBox(currentImage);

        }

        private void UpdatePictureBox(Image img)
        {
            if (pictureBox_Main.InvokeRequired)
            {
                MethodInvoker mi = delegate
                {
                    pictureBox_Main.Image = img;
                    pictureBox_Main.Refresh();
                };
                pictureBox_Main.Invoke(mi);
            }
            else {  //Otherwise (when the calculation is perfomed by the GUI-thread itself) no invoke necessary
                pictureBox_Main.Image = img;
                pictureBox_Main.Refresh();
            }
            img.Dispose();
        }

    }
}

Thanks in advance for further hints :)

Answer 1

SOLVED:

The issue seems to be, that event is fired immediately but the file is not yet finally copied. That means we have to wait until the file is free. A Thread.Sleep(100) at the start of the event does the job. As I now know what to google for, I found two links: This and this where you can find:

The OnCreated event is raised as soon as a file is created. If a file is being copied or transferred into a watched directory, the OnCreated event will be raised immediately, followed by one or more OnChanged events

So, what works best for my case, was to include a method to test if the file is still locked and than wait at the beginning of the event for an unlock of the file. No need for an additional thread or a BackgroundWorker. See the code:

private void OnNewFileInDir(object source, FileSystemEventArgs evtArgs)
{
   Console.WriteLine("New file detected: " + evtArgs.FullPath);

   //Wait for the file to be free
   FileInfo fInfo = new FileInfo(evtArgs.FullPath);
   while (IsFileLocked(fInfo))
   {
       Console.WriteLine("File not ready to use yet (copy process ongoing)");
       Thread.Sleep(5);  //Wait for 5ms
   }

   //Set the current filepath in the class with path of the file added to the folder:
   imageFilepath = evtArgs.FullPath;
   //Load the actual image:
   Image currentImage = Image.FromFile(imageFilepath);    
   UpdatePictureBox(currentImage);
}

private static bool IsFileLocked(FileInfo file)
{
    FileStream stream = null;
    try
    {
        //try to get a file lock
        stream = file.Open(FileMode.Open, FileAccess.ReadWrite, FileShare.None);
    }
    catch (IOException)
    {
       //File isn't ready yet, so return true as it is still looked --> we need to keep on waiting
       return true;
    }
    finally
    {
        if (stream != null){ 
            stream.Close();
            stream.Dispose();
        }
    }
    // At the end, when stream is closed and disposed and no exception occured, return false --> File is not locked anymore
    return false;
}

Nevertheless: Thanks for your help...it got me on the right track;)

Answer 2

As MSDN says about FileSystemWatcher:

Common file system operations might raise more than one event. For example, when a file is moved from one directory to another, several OnChanged and some OnCreated and OnDeleted events might be raised. Moving a file is a complex operation that consists of multiple simple operations, therefore raising multiple events. Likewise, some applications (for example, antivirus software) might cause additional file system events that are detected by FileSystemWatcher.

Maybe your Image is loaded severals times.

To test it, you can add this line after imageFilepath = evtArgs.FullPath;

imageFilepath = evtArgs.FullPath;
Task.Run(()=>{MessageBox.Show(imageFilepath);});

This will inform you about the fact that the Created event is fired, and will not hold up your program.

Edit

Put your line of code that give the OutOfMemory in a Try Catch. Like this and this questions describes, you can get this error if your image is corrupt.