How can I clear the PooledBufferManager for my used once WCF HTTP client?

Question

I have a WCF client application which makes a single service call with a very large response (1GB). I'm finding making this service call uses a lot of memory (500MB) which seems to never get reclaimed even though the response objects are no longer referenced by my code.

I've used a memory profiler to see much of the memory usage lies in byte arrays created by PooledBufferManager instances.

The proxy/client I am using has been auto generated by Visual Studio, so it is a class derived from System.ServiceModel.ClientBase< TChannel >.

I am using a custom binding with the following configuration:

  <customBinding>
    <binding name="foo"
             closeTimeout="00:01:00"
             openTimeout="00:01:00"
             receiveTimeout="00:01:00"
             sendTimeout="00:01:00">
      <transactionFlow/>
      <reliableSession ordered="true" inactivityTimeout="00:02:00"/>
      <security authenticationMode="SecureConversation" messageSecurityVersion="WSSecurity11WSTrustFebruary2005WSSecureConversationFebruary2005WSSecurityPolicy11BasicSecurityProfile10">
        <secureConversationBootstrap messageSecurityVersion="WSSecurity11WSTrustFebruary2005WSSecureConversationFebruary2005WSSecurityPolicy11BasicSecurityProfile10">
          <localClientSettings maxClockSkew="23:59:00"/>
        </secureConversationBootstrap>
        <localClientSettings maxClockSkew="23:59:00"/>
      </security>
      <mtomMessageEncoding>
        <readerQuotas maxDepth="2147483647" maxStringContentLength="2147483647" maxArrayLength="2147483647" maxBytesPerRead="2147483647" maxNameTableCharCount="2147483647"/>
      </mtomMessageEncoding>
      <httpTransport maxBufferPoolSize="2147483647" maxBufferSize="2147483647" maxReceivedMessageSize="2147483647"/>
    </binding>
  </customBinding>

Reading around people talk about a few solutions to this:

switching to streaming response mode instead of buffered, but I believe this is for TCP WCF connections, not for HTTP like mine.
setting maxBufferPoolSize to zero to disable buffer pools. Setting this in the httpTransport element doesn't seem to have any effect for me.
calling Clear() on the appropriate BufferManager/PooledBufferManager. I can't find the object to call this on from the context of the ClientBase derived instance I have. I did manage to find the appropriate PooledBufferManager instance using the debugger and digging many levels deep into private innerChannelFactory fields, but this isn't useful to interact with it from code.
manually invoking GC.Collect() seems to reclaim about 50MB of the outstanding 500MB.

What would be the best way to reclaim as much of this memory as possible used by this one time service call? I'm on the verge of making the service call in a dedicated process I can kill to reclaim the memory at this point.

score 2 · Answer 1 · edited May 23 '17 at 10:33

switching to streaming response mode instead of buffered, but I believe this is for TCP WCF connections, not for HTTP like mine.

Even with http, you can in fact switch to streamed mode as I did for https, but http does work just as well.

Just add a transferMode="Streamed" attribute to your httpTransport element. As you are concerned about the client, you need to do this in the app.config of your client. (You can do it independently in the server's web.config as well, if you also want to change the server to streamed mode. But there's no need to change both client and server, the transfer mode does not change the bytes on the wire)

setting maxBufferPoolSize to zero to disable buffer pools. Setting this in the httpTransport element doesn't seem to have any effect for me.

Which is exactly what this article claims should work:

If maxBufferPoolSize = 0, then a GCBufferManager gets created, otherwise you get a PooledBufferManager. The former is trivial, and in fact doesn't really do any management whatsoever, but simply allocates a new buffer for any request, and lets the garbage collector take care of disposing it.

i.e. a manual GC.Collect() could actually do the trick in this case.

calling Clear() on the appropriate BufferManager/PooledBufferManager. I can't find the object to call this on from the context of the ClientBase derived instance I have.

I wasn't able to access that the BufferManager instance either, back when I tried that.

manually invoking GC.Collect() seems to reclaim about 50MB of the outstanding 500MB.

Won't work for the pooled buffer manager, as it will never release a buffer once created, unless you call Clear(), which you cannot, lacking a pointer to the instance.

How can I clear the PooledBufferManager for my used once WCF HTTP client?

1 Answers1