.Net zlib inflate with .Net 4.5

Question

According to MSDN in .Net 4.5 System.IO.Compression is based on zlib.
I am trying now to change my current interop based reading from a zlib deflated stream from a non .NET server into a BCL based implementation.
My implementation looks like this:

    var enc = new UTF8Encoding();            
        var readBytes = BufferSizeRaw;
        var outputBuffer = new byte[BufferSizeRaw];            
        var networkBuffer = _networkQueue.Take();
        var ms = new MemoryStream(networkBuffer.InputBuffer, 0, networkBuffer.UsedLength);
        using (Stream stream = new DeflateStream(ms, CompressionMode.Decompress))
            while (readBytes==BufferSizeRaw)
            {
                readBytes = stream.Read(outputBuffer, 0, outputBuffer.Length);                
                stringBuffer+= enc.GetString(outputBuffer, 0, readBytes);                
            }

I receive the following exception on the first call of the decompression/read on the DeflateStream :

Block length does not match with its complement

The interop based call uses var result=inflate(ref zStyream, ZLibFlush.NoFlush;
Has anyone tried the same or sees a reason for an error in the code or is there a wrong understanding on my end? I have also tried it with truncating the first two bytes without any luck.
The first few bytes are 20, 202, 177,13.

score 19 · Answer 1 · edited May 23 '17 at 11:54

The answer above is correct but isn't exactly clear on the "why". The first two bytes of a raw ZLib stream provide details about the type of compression used. Microsoft's DeflateStream class in System.Io.Compression doesn't understand these. The fix is as follows:

using (MemoryStream ms = new MemoryStream(data))
{
    MemoryStream msInner = new MemoryStream();

    // Read past the first two bytes of the zlib header
    ms.Seek(2, SeekOrigin.Begin);

    using (DeflateStream z = new DeflateStream(ms, CompressionMode.Decompress))
    {
        z.CopyTo(msInner);

In this example, data is a Byte[] with the raw Zlib file. After loading it into a MemoryStream, we simply "seek" past the first two bytes.

The post explains what a Zlib header looks like if you are looking at the raw bytes.

What does a zlib header look like?

score 9 · Accepted Answer · answered Jun 20 '13 at 19:46

9

What are the first several bytes of the data you are trying to compress?

You might have zlib, gzip, or raw deflate data that you're trying to decode.

By the way, I highly recommend that you use DotNetZip's interface to zlib instead of NET 4.5's (or NET any version). NET 4.5 has bugs in that interface that Microsoft has declared that they won't fix (!).

answered Jun 20 '13 at 19:46

Mark Adler

101,978
13
118
158

There was another error in my program, which I have now resolved and it is working now. My understanding is that a binary zlib version is used since version 4.5 - that it is the background why I did the switch. I was using the native version before for performance reasons. – weismat Jun 21 '13 at 08:19
Need to correct myself - it works just for a few reads - most likely it is related to the memory stream. Will investigate it further. – weismat Jun 21 '13 at 08:42
While they did finally switch to using zlib in 4.5, which fixed a bunch of really brain-dead problems, the interface to zlib is messed up and they won't fix it. So I recommend going back to native or using DotNetZip. – Mark Adler Jun 21 '13 at 15:02
@MarkAdler Although DotNetZip is the best in error detection, it's about twice slower than System.IO.Compression.DeflateStream and three times slower than SharpZipLib in some of my real life scenarios. But SharpZipLib still has a weird bug of endless inflating of some broken packets. – tsul Aug 05 '16 at 12:09
Do you know if you're comparing apples and apples? Changing compression settings can result in greater still speed differences. – Mark Adler Aug 06 '16 at 07:54
Apparently [as of .NET 4.7.2](https://blogs.msdn.microsoft.com/dotnet/2018/04/30/announcing-the-net-framework-4-7-2/) those bugs(or 1 big bug anyway) have been fixed, though you have to opt in. – SensorSmith May 04 '18 at 20:49
@SensorSmith Opt in? – Mark Adler May 04 '18 at 21:27
@Mark Adler Either you have to target .NET 4.7.2 or make an entry in 'App.config'(or similar) for ZLib to be used for decompression. (See the original link.) – SensorSmith May 04 '18 at 21:37
6 years later the link to the bugs MS doesn't want to fix 404s, is it still valid? Should I abstain from using System.IO.Compression for zlib streams? – Benni Aug 30 '19 at 18:44
1

@Benni See comments above. 4.7.2 allegedly fixes the referenced bug. – Mark Adler Aug 30 '19 at 23:55

score 5 · Answer 3 · answered Apr 18 '14 at 15:21

Here's a revised version of ProVega's answer, which inflates the byte array into a string:

using (var stream = new MemoryStream(bytes,2, bytes.Length - 2))
using (var inflater = new DeflateStream(stream, CompressionMode.Decompress))
using (var streamReader = new StreamReader(inflater))
{
    return streamReader.ReadToEnd();
}

.Net zlib inflate with .Net 4.5

3 Answers3

Linked