Decode Base64 and Inflate Zlib compressed XML

Question

Sorry for the long post, will try to make this as short as possible.

I'm consuming a json API (which has zero documentation of course) which returns something like this:

{
uncompressedlength: 743637,
compressedlength: 234532,
compresseddata: "lkhfdsbjhfgdsfgjhsgfjgsdkjhfgj"
}

The data (xml in this case) is compressed and then base64 encoded data which I am attempting to extract. All I have is their demo code written in perl to decode it:

use Compress::Zlib qw(uncompress);
use MIME::Base64  qw(decode_base64);
my $uncompresseddata = uncompress(decode_base64($compresseddata));

Seems simple enough.

I've tried a number of methods to decode the base64:

 private string DecodeFromBase64(string encodedData)
    {
        byte[] encodedDataAsBytes = System.Convert.FromBase64String(encodedData);
        string returnValue = System.Text.Encoding.Unicode.GetString(encodedDataAsBytes);
        return returnValue;
    }

    public string base64Decode(string data)
    {
        try
        {
            System.Text.UTF8Encoding encoder = new System.Text.UTF8Encoding();
            System.Text.Decoder utf8Decode = encoder.GetDecoder();

            byte[] todecode_byte = Convert.FromBase64String(data);
            int charCount = utf8Decode.GetCharCount(todecode_byte, 0, todecode_byte.Length);
            char[] decoded_char = new char[charCount];
            utf8Decode.GetChars(todecode_byte, 0, todecode_byte.Length, decoded_char, 0);
            string result = new String(decoded_char);
            return result;
        }
        catch (Exception e)
        {
            throw new Exception("Error in base64Decode" + e.Message);
        }
    }

And I have tried using Ionic.Zip.dll (DotNetZip?) and zlib.net to inflate the Zlib compression. But everything errors out. I am trying to track down where the problem is coming from. Is it the base64 decode or the Inflate?

I always get an error when inflating using zlib: I get a bad Magic Number error using zlib.net and I get "Bad state (invalid stored block lengths)" when using DotNetZip:

string decoded = DecodeFromBase64(compresseddata);
string decompressed = UnZipStr(GetBytes(decoded));


 public static string UnZipStr(byte[] input)
    {

        using (MemoryStream inputStream = new MemoryStream(input))
        {
            using (Ionic.Zlib.DeflateStream zip =
              new Ionic.Zlib.DeflateStream(inputStream, Ionic.Zlib.CompressionMode.Decompress))
            {
                using (StreamReader reader =
                  new StreamReader(zip, System.Text.Encoding.UTF8))
                {
                    return reader.ReadToEnd();
                }
            }
        }
    }

After reading this: http://george.chiramattel.com/blog/2007/09/deflatestream-block-length-does-not-match.html And listening to one of the comments. I changed the code to this:

MemoryStream memStream = new MemoryStream(Convert.FromBase64String(compresseddata));
memStream.ReadByte();
memStream.ReadByte();
DeflateStream deflate = new DeflateStream(memStream, CompressionMode.Decompress);
string doc = new StreamReader(deflate, System.Text.Encoding.UTF8).ReadToEnd();

And it's working fine.

FYI, you should never use `throw new Exception("message" + e.Message);`. You have no idea what's in `e.Message` so you don't know how it will look concatenated with your message. More importantly, you should always include the original exception in the `innerException` parameter, so `throw new Exception("message", e);` — John Saunders, Mar 12 '13 at 01:03
That exception is just for the debugging process. It will be disappeared soon. But thanks — Paul Perrick, Mar 12 '13 at 02:27
There's no reason to indulge in bad habits just because it's debugging code. — John Saunders, Mar 12 '13 at 03:02

score 3 · Answer 1 · answered Mar 12 '13 at 02:58

This was the culprit:

http://george.chiramattel.com/blog/2007/09/deflatestream-block-length-does-not-match.html

With skipping the first two bytes I was able to simplify it to:

MemoryStream memStream = new MemoryStream(Convert.FromBase64String(compresseddata));
memStream.ReadByte();
memStream.ReadByte();
DeflateStream deflate = new DeflateStream(memStream, CompressionMode.Decompress);
string doc = new StreamReader(deflate, System.Text.Encoding.UTF8).ReadToEnd();

score 1 · Answer 2 · answered Mar 12 '13 at 01:08

1

First, use System.IO.Compression.DeflateStream to re-inflate the data. You should be able to use a MemoryStream as the input stream. You can create a MemoryStream using the byte[] result of Convert.FromBase64String.

You are likely causing all kinds of trouble trying to convert the base64 result to a given encoding; use the raw data directly to Deflate.

answered Mar 12 '13 at 01:08

Joe

41,484
20
104
125

1

Thank you for your comment. I am probably really over thinking this. If I use something like this: DeflateStream deflate = new DeflateStream(new MemoryStream(Convert.FromBase64String(compresseddata)), CompressionMode.Decompress); string doc = new StreamReader(deflate, System.Text.Encoding.UTF8).ReadToEnd(); I get the error: Block length does not match with its complement. I'm starting to wonder if the original document is formatted properly. – Paul Perrick Mar 12 '13 at 02:36

Decode Base64 and Inflate Zlib compressed XML

2 Answers2

Linked