Get byte[] from dataset and compress

Question

I am returning a custom class from a WCF operation. The binding used is netTcp. This custom class contains several data members. One of them is a dataset. The dataset can be huge depending on particular actions. I am planning to compress the dataset to bytes and then return the custom class back.

Based on the reading I have come up with following code to return compressed bytes from a dataset. But not sure if this is the best way (or the correct way) to go. Your thoughts pls. ??

            byte[] bytes = null;
            byte[] compressedBytes = null;
            using(var memory = new MemoryStream())
            {
                var formatter = new BinaryFormatter();
                formatter.Serialize(memory, ds);
                bytes = memory.ToArray();
            }

            using(var memory = new MemoryStream())
            {
                using(var gzip = new GZipStream(memory, CompressionMode.Compress, true))
                {
                    gzip.Write(bytes, 0, bytes.Length);
                    compressedBytes = memory.ToArray();
                }
            }

            return compressedBytes;    

Answer 1

There is an important step to save space:

ds.RemotingFormat = SerializationFormat.Binary;

Otherwise it internally uses xml, even via BinaryFormatter. With this in place, you can also include gzip, but the gain isn't quite as significant. As it happens I have some stats to compare this here; to copy that data in:

DataTable (xml) (vanilla)               2269ms/6039ms
                                       64,150,771 bytes
DataTable (xml) (gzip)                  4881ms/6714ms
                                       7,136,821 bytes
DataTable (xml) (deflate)               4475ms/6351ms
                                       7,136,803 bytes
BinaryFormatter (rf:binary) (vanilla)   2006ms/3366ms
                                       11,272,592 bytes
BinaryFormatter (rf:binary) (gzip)      3332ms/4267ms
                                       8,265,057 bytes
BinaryFormatter (rf:binary) (deflate)   3216ms/4130ms

But: DataSet is not a very WCF way of doing things. I would add standard OO classes, and swap the serializer for something like protobuf-net, which is significantly smaller than either DataContractSerializer or NetDataContractSerializer.

Answer 2

You can just use the following:

        using(var memory = new MemoryStream())
        {
            using(var gzip = new GZipStream(memory, CompressionMode.Compress, true))
            {
                var formatter = new BinaryFormatter();
                formatter.Serialize(gzip, ds);
            }

            return memory.ToArray();
        }