1

I need to be able to compress a data stream with PowerShell and decompress it with my C program using zlib.

I'm using System.IO.Compression.DeflateStream in PowerShell to compress, but the decompression with zlib in my C program does not work.

Apparently DeflateStream adds extra data/header to the output. If that's right, I want to know its format so I can implement a decompression with zlib.

karliwson
  • 3,365
  • 1
  • 24
  • 46

1 Answers1

2

To answer my own question, it's actually the opposite.

System.IO.Compression.DeflateStream doesn't add any extra data, it outputs a raw deflate stream with no header/metadata.

I actually had to add the ZLib header to the stream. See RFC 1950.

This answer also helped me:

What does a zlib header look like?

In detail, there must be two bytes in a zlib header:

0   1
+---+---+
|CMF|FLG|
+---+---+

First byte (CMF):

Each nibble (half-byte) of the first byte has a meaning:

bits 0 to 3  CM     Compression method
bits 4 to 7  CINFO  Compression info

You can see it in more detail in the answer I've mentioned above.

Second byte (FLG):

FLG (FLaGs) This flag byte is divided as follows:

bits 0 to 4  FCHECK  (check bits for CMF and FLG)
bit  5       FDICT   (preset dictionary)
bits 6 to 7  FLEVEL  (compression level)

The most common headers are:

78 01 - No Compression/low
78 9C - Default Compression
78 DA - Best Compression

I've added the header 78 9C before System.IO.Compression.DeflateStream's output, but there's one more step as @Mark Adler said in the comments: we have to add an Adler-32 checksum at the end of the stream.

Edit:

In the end, I was trying to reinvent the wheel. I can use System.IO.Compression.GZipStream in the PowerShell compression and just use inflateInit2() in the decompression and all should be ok (thanks to @Mark Adler's suggestion).

Community
  • 1
  • 1
karliwson
  • 3,365
  • 1
  • 24
  • 46
  • It can't "work like a charm" if you don't also add the check value at the end. `inflate()` will not return with `Z_STREAM_END` without that, which is what indicates success. You will have no idea whether your stream was incomplete or not if you just accept `Z_OK`. You don't need to add the the zlib header anyway. You can use `inflateInit2()` to request a raw inflate, in which case it will return `Z_STREAM_END` if complete. Also I would recommend that you use `GZipStream` instead to provide a proper wrapper with integrity checking. You can use `inflateInit2()` to request gzip decoding. – Mark Adler Oct 09 '16 at 14:46