1

I am using BufferManager in my WCF service. I created my own class to wrap around the BufferManager which implements IDisposable. Right now my Dispose method looks like this:

public void Dispose()
    {
        this.bufferManager.Clear();
    }

My question: does this accomplish the same thing as calling ReturnBuffer on all of the buffers that have been taken by the manager?

Just for some background: I am using the BufferManager in the following method:

public byte[] ReadAllBufferedBytes(string filePath)
    {
        using (var fileStream =
            new FileStream(filePath, FileMode.Open, FileAccess.Read))
        {
            byte[] buffer = this.bufferManager.TakeBuffer((int)fileStream.Length);

            fileStream.Read(buffer, 0, buffer.Length);

            return buffer;
        }
    }

The reason I am doing this is because I kept getting OutOfMemory exceptions which would tear down the service.

The host server has 3 GB of memory. The service is in InstanceContextMode.Single mode, so images are processed one at a time. The images are received as byte arrays - the biggest might be 100MB, but are typically much smaller - converted, and then returned as a byte array. A lot ends up on the Large Object Heap, and image sizes vary quite a bit.

I am wondering if the issue is heap fragmentation.

As each page of a document gets converted, it is appended to a temp file on disk. After the conversion, I read the entire converted file from disk into a byte array and return it to the client.

The standard File.ReadAllBytes method creates a new byte array when it reads from the file, which inevitably ends up on the LOH due to the image sizes I'm working with (I assume this is what happens). I created the ReadAllBufferedBytes method to do the same thing, but to buffer the byte array and let the BufferManager return the buffer when it is disposed.

Another question is: do I even need to do all this?

lintmouse
  • 5,079
  • 8
  • 38
  • 54

1 Answers1

0

The BufferManager is normally used in scenarios where you must prevent GC pressure - where there are a lot of small byte[] assignments, like when receiving or sending data on a very low level (e.g. sockets). The emphasis here is on a lot, otherwise the GC should be able to handle the memory allocations just fine.

To prevent the loading of the entire converted document file into memory you should use the FileStream directly (without reading it's entire content into memory - a byte[]) in combination with the streamed (response) TransferMode, if possible.

m0sa
  • 10,712
  • 4
  • 44
  • 91
  • If I change my operation contract to a streamed transfer mode, I would have to update the clients to consume the stream. It will take some wrestling with my team mates to get them on board with that idea. My main concern with the current implementation is LOH fragmentation because all of the byte[]s are > 85kb. When an operation contract receives a byte[], does WCF internally manage it with a buffer manager? – lintmouse Feb 04 '13 at 15:32
  • Sure, that's why the BufferManager was created. In buffered transfer mode the parameters for method calls are deserialized only after the complete SOAP message has been received. So you will always have to load the entire file into memory if you don't use streamed transfer. – m0sa Feb 04 '13 at 16:33