For detecting compression formats, see : Detecting Compression Formats
You can determine that it is likely to be one of those formats by
looking at the first few bytes. You should then test to see if it
really is one of those, using an integrity check from the associated
utility for that format, or by actually proceeding to decompress.
You can find the header formats in the descriptions:
Zip (.zip) format description, starts with 0x50, 0x4b, 0x03, 0x04
(unless empty — then the last two are 0x05, 0x06 or 0x06, 0x06) Gzip
(.gz) format description, starts with 0x1f, 0x8b, 0x08 xz (.xz) format
description, starts with 0xfd, 0x37, 0x7a, 0x58, 0x5a, 0x00 Others:
zlib (.zz) format description, starts with (in bits) 0aaa1000
bbbccccc, where ccccc is chosen so that the first byte times 256 plus
the second byte is a multiple of 31. compress (.Z) starts with 0x1f,
0x9d bzip2 (.bz2) starts with 0x42, 0x5a, 0x68
For unziping files in C#, see: Uncompressing Files in .NET
We have used SharpZipLib successfully on many projects. I know it's a
third party tool, but source code is included and could provide some
insight if you chose to reinvent the wheel here.
There are a few different answers on the above link that may provide a better path depending on your needs.