Memory leak when COM interface passed in COM callback to .Net app

Question

I am attempting to use the Microsoft Smooth Streaming Format SDK to implement basically a CCTV system. The SDK has a sample application that is in C++ but I am writing in C#. I have the code ported and it works correctly except for an apparent memory leak in a COM->.NET callback that passes a COM object.

ETA: The memory leak was measured with the trial version of RedGate ANTS Memory Profiler 8.10. It shows a steady climbing Private Bytes and Working Set - Private graph. Also, the "Unmanaged memory breakdown by module" shows that "CLR (estimated)" is continually growing. In a test left running overnight, it grew 332MB in ~16hrs. While this is a pretty slow growth rate, it will eventually cause an OutOfMemory situation. As stated before this app is supposed to stream 24x7.

The COM object is a custom DirectShow sample grabber filter. The SSF SDK has a sample grabber filter implementation but it exhibits this same behavior and is not to be redistributed anyway.

The sample grabber filter IDL is as such:

[
  object,
  uuid(17b2823e-a24b-483f-a0b0-002edaf56035),
  helpstring("ISampleGrabberCB interface"),
  pointer_default(unique)
]
interface ISampleGrabberCallback : IUnknown
{
   HRESULT OnSample([in] IMediaSample *pSample);
}


 [
   object,
   uuid(9495f2d0-35fd-451f-b831-f89f1af8589f),
   dual,
   helpstring("ISampleGrabberFilter Interface"),
   pointer_default(unique)
 ]
 interface ISampleGrabberFilter : IDispatch
 {
    HRESULT SetCallback([in] ISampleGrabberCallback *callbackIf);
 };

 [
    uuid(2bd7d268-bb30-4a6f-b715-6223c006d973),
    helpstring("SampleGrabber Class")
 ]
 coclass SampleGrabberFilter
 { 
    [default] interface ISampleGrabberFilter;
 };

IMediaSample is defined by DirectShow in strmif.idl and is imported into the IDL file.

I define a COM Interop file (which uses DirectShowLib.Net for the definition of IMediaSample) like so:

using System;
using System.Runtime.CompilerServices;
using System.Runtime.InteropServices;

using DirectShowLib;

namespace Interop.SampleGrabber
{

   [Guid("9495F2D0-35FD-451F-B831-F89F1AF8589F")]
   [TypeLibType(TypeLibTypeFlags.FDual | TypeLibTypeFlags.FDispatchable)]
   [ComImport]
   public interface ISampleGrabberFilter
   {
      [DispId(1610743808)]
      [MethodImpl(MethodImplOptions.InternalCall, MethodCodeType = MethodCodeType.Runtime)]
      int SetCallback([MarshalAs(UnmanagedType.Interface), In] ISampleGrabberCallback callbackIf);
   }

   [Guid("17B2823E-A24B-483F-A0B0-002EDAF56035")]
   [InterfaceType(ComInterfaceType.InterfaceIsIUnknown)]
   [ComImport]
   public interface ISampleGrabberCallback
   {
      [MethodImpl(MethodImplOptions.InternalCall, MethodCodeType = MethodCodeType.Runtime)]
      [PreserveSig]
      int OnSample([MarshalAs(UnmanagedType.Interface), In] IMediaSample pSample);
   }

   [CoClass(typeof(SampleGrabberFilterClass))]
   [Guid("9495F2D0-35FD-451F-B831-F89F1AF8589F")]
   [ComImport]
   public interface SampleGrabberFilter : ISampleGrabberFilter
   {
   }

   [TypeLibType(TypeLibTypeFlags.FCanCreate)]
   [ClassInterface(ClassInterfaceType.None)]
   [Guid("2BD7D268-BB30-4A6F-B715-6223C006D973")]
   [ComImport]
   public class SampleGrabberFilterClass : ISampleGrabberFilter, SampleGrabberFilter
   {
      [DispId(1610743808)]
      [MethodImpl(MethodImplOptions.InternalCall, MethodCodeType = MethodCodeType.Runtime)]
      [PreserveSig]
      public extern virtual int SetCallback([MarshalAs(UnmanagedType.Interface), In] ISampleGrabberCallback callbackIf);
   }

}

The COM interop file was made by tweaking the decompilation of a TlbImp-generated interop dll with DotPeek 2016.1 (in order to use the DirectShowLib.NET version of IMediaSample). The custom interop file is "linked" directly in the program; i.e. it is not a separate assembly.

The C++ code which makes the callback looks like this:

// in SampleGrabberFilter.h file
CComPtr<ISampleGrabberCallback> callback;

// in SampleGrabberFilter.cpp file
HRESULT SampleGrabberFilter::SetCallback(ISampleGrabberCallback *cb)
{
   callback = cb;
   return S_OK;
}


HRESULT SampleGrabberFilter::DoRenderSample(
   IMediaSample *pSample)
{
   HRESULT hr = S_OK;
   if (callback != nullptr)
   {
      hr = callback->OnSample(pSample);
   }
   pSample->Release();
   return hr;
}

And the C# code which implements the callback looks like this:

public class StreamContext : Interop.SampleGrabber.ISampleGrabberCallback, 
{
  // ...

  int OnSample_(IMediaSample sample)
  {
     Console.WriteLine("OnSample_ : Size = {0}", sample.GetSize());
     if (sample == null)
        return 0;

     int hr = 0; 
     hr = ProcessSample(sample);
     //Marshal.Release(Marshal.GetIUnknownForObject(sample));
     Marshal.ReleaseComObject(sample);
     //int n = Marshal.FinalReleaseComObject(sample);
     GC.Collect();
     return hr;
  }

  //...
 }

Note all my different attempts to release the IMediaSample.

This code all works great except there appears to be a leak if I pass an actual IMediaSample to the callback method; even if I comment out the call to ProcessSample above effectively making the callback a NOOP. However, if I pass nullptr to the callback, there is no leak.

So something seems to be AddRef'ing the IMediaSample and not releasing it.

What am I missing here?

Update: I've added another method to the callback interface that passes the buffer from the IMediaSample (and other info) instead of the entire IMediaSample like so:

HRESULT OnSampleBuffer(REFERENCE_TIME startTime, REFERENCE_TIME endTime, int bufferLen, LPBYTE buffer, BOOL isSyncPoint)

When this form of callback is made, there is no memory leak.

Answer 1

I've this comment in a code of mine that works with COM objects with events from C# application

        // Call the Garbage Collector twice (only once is not enough)
        GC.Collect();
        GC.WaitForPendingFinalizers();
        GC.Collect();
        GC.WaitForPendingFinalizers();

There's a question in SO about why Under what circumstances, we need to call GC.Collect twice