wcf conditional compression

Question

I recently plugged in a custom encoder (uses binary encoder to do the actual encoding and Gzip compresser to compress the byte array). It works fine. The issue now is for small message size it actually inflates the byte array. I would like to know if there is a way to avoid this. Specifically if there is a way I can apply condition compression and decompression.

I did try to do something like - place a condition

if(buffer.Count <= 5000)
 skip compression

But the problem is on the other end decompression will happen even if the bytes are not compressed. I hope this makes sense.

Following are the functions where compression and decompression happens (code snippet from CompactMessageEncoder)

public override Message ReadMessage(ArraySegment<byte> buffer, BufferManager bufferManager, string contentType)
        {

            ArraySegment<byte> decompressedBuffer = DecompressBuffer(buffer, bufferManager);
            LogWrite("Decompressed from {0} bytes to {1} bytes", buffer.Count, decompressedBuffer.Count);

            Message returnMessage = _innerEncoder.ReadMessage(decompressedBuffer, bufferManager);

            returnMessage.Properties.Encoder = this;
            return returnMessage;
        }



public override ArraySegment<byte> WriteMessage(Message message, int maxMessageSize, BufferManager bufferManager, int messageOffset)
        {
            var buffer = _innerEncoder.WriteMessage(message, maxMessageSize, bufferManager, messageOffset);

            var compressedBuffer = CompressBuffer(buffer, bufferManager, messageOffset);
            LogWrite("Compressed from {0} bytes to {1} bytes", buffer.Count, compressedBuffer.Count);

            return compressedBuffer;
        }

could you add a custome header to those messages that were compressed? http://stackoverflow.com/questions/964433/how-to-add-a-custom-header-to-every-wcf-calls — Mitch Wheat, Jan 08 '11 at 01:15
@Mitch: This might not work. Since I will have to add Lets say I add custom header to the Message class and compress it in WriteMessage() function. In the ReadMessage() I will have to decompress it before getting the Message class and reading the header. — stackoverflowuser, Jan 11 '11 at 21:37

score 2 · Answer 1 · edited May 23 '17 at 11:51

In case you are using http and if you are on IIS 7 or higher (it's also possible with 6, but configuration is harder apparently), you could use the builtin gzip / deflate compression.

See http://www.iis.net/ConfigReference/system.webServer/httpCompression

That link also explains parameters like 'minFileSizeForComp' which exactly addresses your issue. It also has a lot of other nice parameters like 'dynamicCompressionDisableCpuUsage', which will disable compression once a certain cpu load is exceeded.

The nice thing about http compression is that it is a standard and client and server can determine per request whether they can or want to compress using standard http request and response headers. Also, the compression format can be changed from gzip to deflate, the latter having a couple of advantages, despite not being used as much. See http://madskristensen.net/post/Compression-and-performance-GZip-vs-Deflate.aspx and http://www.vervestudios.co/projects/compression-tests/

One potential downside - depending on your app, is that http compression is done only from server -> client, so if your requests are huge, that might become an issue, but in probably most cases, server responses will be way bigger than client requests. Edit: If you are willing to add a custom IHttpModule, you can even get request compression working.

I just successfully migrated a heavily used farm of 20 servers from GZIPMessageEncoder to IIS 7.5's http compression.

On a side note, please do not use the MS' GzipMessageEncoder example in buffered transfer-mode, which is the default. GzipMessageEncoder just wastes tons of memory - hundreds of megabytes in my particular case, see my answer here: WCF HttpTransport: streamed vs buffered TransferMode

score 0 · Answer 2 · answered Jan 08 '11 at 01:21

Could you write a byte that would indicate whether the data was compressed or not? 0 = uncompressed, 1 = Gzip compressed, 2 - 255 = Future compression algorithms?

You could write the uncompressed length, but that would rely on the server and client using the same cutoff length for not compressing. You could write a Boolean indicating whether it is compressed, but that would still take a whole byte, and you can leave yourself open for future expansion by writing a value 0-255. (Maybe you'll discover that with your data, a different compression algorithm can give even better compression.)

wcf conditional compression

2 Answers2

Linked