Running out of memory looping through mail items

Question

Hi I have a Outlook com addin that is doing some simple searching tricks for me. I am part way through putting it together but I am having issues with it running out of memory. The process is very simple and basically loops through an outlook folder checking each mailItem for a match. given the loop reinitialize the variables each time I would have expected the garbage collector to keep up but when I watch the memory it loses ~10m/sec until the system is out of memory and I get unhandled exceptions.

This is part of the code

private void FindInFolder(Outlook.MAPIFolder FolderToSearch)
    {
        Outlook.MailItem mailItem;
        Outlook.MAPIFolder ParentFolder;

        int counter = 0;

        StatusBar.Text = "Searching in Folder " + FolderToSearch.FolderPath + "/" + FolderToSearch.Name;
        StatusBar.Update();
        this.Update();

        foreach (COMObject item in FolderToSearch.Items)
        {
            counter++;
            if (counter % 100 == 0)
            {
                StatusBar.Text = FolderToSearch.FolderPath + "/" + FolderToSearch.Name + " item " + counter + " of " + FolderToSearch.Items.Count;
                StatusBar.Update();
                if (counter % 1000 == 0)
                {
                    GC.Collect();
                }
            }
            if (item is Outlook.MailItem)
            {
                mailItem = item as Outlook.MailItem;
                if (IsMatch(mailItem))
                {
                    if (mailItem.Parent is Outlook.MAPIFolder)
                    {
                            ParentFolder = mailItem.Parent as Outlook.MAPIFolder;
                            ResultGrd.Rows.Add(mailItem.EntryID, ParentFolder.FolderPath, mailItem.SenderName, mailItem.Subject, mailItem.SentOn);
                    }
                }
            }
            mailItem = null;
        }
    }

Which calls

        private Boolean IsMatch(Outlook.MailItem inItem)
    {
        Boolean subBool = false;
        Boolean NameBool = false;

        try
        {
            if (null != inItem)
            {
                if (SubjectTxt.Text != "")
                {
                    if (inItem.Subject.Contains(SubjectTxt.Text))
                    {
                        subBool = true;
                    }
                }
                else
                {
                    subBool = true;                    
                }

                if (NameTxt.Text != "")
                {
                    if (inItem.Sender != null)
                    {
                        if (inItem.Sender.Name.Contains(NameTxt.Text))
                        {
                            NameBool = true;
                        }
                    }
                }
                else 
                {
                    NameBool = true;
                }

                return subBool && NameBool;

            }
        }
        catch (System.Runtime.InteropServices.COMException ce)
        {
            if (ce.ErrorCode == -2147467259)
            {
                //DO nothing just move to the next one
            }
            else
            {
                MessageBox.Show("Crash in IsMatch error code = " + ce.ErrorCode + " " + ce.InnerException);
            }
        }
        return false;
    }

Please excuse all the error catching part at the bottom and the GC.collect they are some of my attempts to work out what is wrong and free up memory.

Note also FindInFolder is called by a new thread so I can interact with results while it continues to search.

What I have tried so far:

Making variables local to function not class so the are retrievable by G, however the most used variable in 'item' as it is part of foreach it must be declared that way.

every 1000 mailItems do a manual GC, this made no difference at all.

For some reason it needs alot of memory just looping through the items and GC never frees them up.

Please also note I am using netoffice not VSTO for Com addin.

Answer 1

When working with COM objects from C#, there were 2 tricks that I used to prevent memory and COM object reference counts from building:

1. Use System.Runtime.InteropServices.Marshal.ReleaseComObject() to release COM objects as you are done with them. This forces a COM "release" on the object.
2. Do not foreach to iterate through a COM collection. foreach holds on to an enumerator object, which prevents other objects from being released.

So, instead of this:

foreach (COMObject item in FolderToSearch.Items)
{
    // ....
}

do this:

Items items = FolderToSearch.Items;
try
{
    for (int i = 0; i < items.Count; ++i)
    {
        COMObject item = items[i];
        try
        {
            // work
        }
        finally
        {
            System.Runtime.InteropServices.Marshal.ReleaseComObject(item);
        }
    }
}
finally
{
    System.Runtime.InteropServices.Marshal.ReleaseComObject(items);
}

These tips helped me reduce memory and object consumption.

Disclaimer: I cannot attest to whether this is good practice or not though.

Answer 2

First of it all: This is NetOffice code and you dont need Marshal.ReleaseComObject in NetOffice. (Moreover, its useless to call ReleaseComObject here) Use Dispose() for the instance instead.

Keep in your mind: NetOffice handle COM proxies for you (Thats why its allowed to use two 2 dots in NetOffice). In your case its internaly stored as: // FolderToSearch -- Items --Enumerator -- Item -- Item -- ....

Use item.Dispose() at the end of each loop to remove/free the item instance or use the following after foreach

FolderToSearch.Dipose() // dispose folder instance and all proxies there comes from

FolderToSearch.DisposeChildInstances() // dispose all proxies there comes from but keep the folder instance alive

Next: The Items enumerator here is a custom enumerator(given by NetOffice) However it works fine with small amount of items but dont do this in a more heavier scenario(may exchange server and thousands of items). The local workstation/program can't handle this in memory. For this reason Microsoft provides only the well kown GetFirst/GetNext pattern. In NetOffice it looks like:

Outlook._Items items = FolderToSearch.Items;
COMObject item = null;
do
{
    if (null == item)
       item = items.GetFirst() as COMObject;
    if (null == item)
       break;

    // do what you want here

    item.Dispose();
    item = items.GetNext() as COMObject;
} while (null != item);

This should works as well.

Answer 3

First, I'd recommend using the Find/FindNext or Restrict methods of the Items class instead of iterating through all items in the folder. For example:

    Sub DemoFindNext() 
     Dim myNameSpace As Outlook.NameSpace 
     Dim tdystart As Date 
     Dim tdyend As Date 
     Dim myAppointments As Outlook.Items 
     Dim currentAppointment As Outlook.AppointmentItem 
 
     Set myNameSpace = Application.GetNamespace("MAPI") 
     tdystart = VBA.Format(Now, "Short Date") 
     tdyend = VBA.Format(Now + 1, "Short Date") 
     Set myAppointments = myNameSpace.GetDefaultFolder(olFolderCalendar).Items 
     Set currentAppointment = myAppointments.Find("[Start] >= """ & tdystart & """ and [Start] <= """ & tdyend & """") 
     While TypeName(currentAppointment) <> "Nothing" 
       MsgBox currentAppointment.Subject 
       Set currentAppointment = myAppointments.FindNext 
     Wend 
    End Sub

See the following articles for more information and sample code:

• How To: Use Find and FindNext methods to retrieve Outlook mail items from a folder (C#, VB.NET)
• How To: Use Restrict method to retrieve Outlook mail items from a folder

Also you may find the AdvancedSearch method of the Application class helpful. The key benefits of using the AdvancedSearch method are listed below:

• The search is performed in another thread. You don’t need to run another thread manually since the AdvancedSearch method runs it automatically in the background.
• Possibility to search for any item types: mail, appointment, calendar, notes etc. in any location, i.e. beyond the scope of a certain folder. The Restrict and Find/FindNext methods can be applied to a particular Items collection (see the Items property of the Folder class in Outlook).
• Full support for DASL queries (custom properties can be used for searching too). You can read more about this in the Filtering article in MSDN. To improve the search performance, Instant Search keywords can be used if Instant Search is enabled for the store (see the IsInstantSearchEnabled property of the Store class).
• You can stop the search process at any moment using the Stop method of the Search class.

Second, I always suggest releasing underlying COM objects instantly. Use System.Runtime.InteropServices.Marshal.ReleaseComObject to release an Outlook object when you have finished using it. Then set a variable to Nothing in Visual Basic (null in C#) to release the reference to the object. You can read more about that in the Systematically Releasing Objects article.

If you want to use GC, you need to call the Collect and WaitForPendingFinalizers methods twice.

Matt, you still don't release all objects in the code. For example:

    for (int i = 0; i < FolderToSearch.Items.Count; ++i)
    {
       COMObject item = FolderToSearch.Items[i];

The Items property of the Folder class returns an instance of the corresponding class which should be released after. I see at lease two lines of code where the reference counter is increased.