6

From MSDN: DeflateStream Class

DeflateStream cannot be used to compress files larger than 4 GB.

Are there any other implementations for .NET without the 4 GB limit?

NOTE: I really need to decompress a file in GZ format with content larger than 4 GB. Can any code do that?

Abel
  • 56,041
  • 24
  • 146
  • 247
Luca Martinetti
  • 3,396
  • 6
  • 34
  • 49

7 Answers7

8

FYI, we have removed the 4 GB limit from DeflateStream in .NET 4.

Justin Van Patten
  • 801
  • 5
  • 4
  • But someone like me who is deflating gzipped files in SSIS is out of luck - you guys skipped releasing a new version with vs2010 / sql2008 r2 :( – Burg Sep 16 '10 at 15:20
7

There is sample code at CodeProject using the 7-Zip library.

The license is open, so you should be able to use this in your project.

7-Zip also supports GZ files.

Peter Mortensen
  • 30,738
  • 21
  • 105
  • 131
benPearce
  • 37,735
  • 14
  • 62
  • 96
2

Take a look at SharpZipLib. Not sure if it's subject to the same limitation, but worth a look.

Dave Markle
  • 95,573
  • 20
  • 147
  • 170
1

Having a look around, it seems a lot of people have encountered this problem. System.IO.Compressio.DeflateStream clarifications please seems to be the most comprehensive.

The only implementation I was able to find that seems to overcome this problem by using Zip64 is Xceed Zip for .NET.

However, it is very expensive and I am not sure if it would suite your needs.

Edit:

There does seem to be quite a number of implementations of Zip64 for .NET, but I can't find any that are free.

Peter Mortensen
  • 30,738
  • 21
  • 105
  • 131
Mark Davidson
  • 5,503
  • 5
  • 35
  • 54
  • That external thread is not helpful. It is filled with supposition, and wrong ones. The DeflateStream does not keep all the stream data in memory - that is not the reason for the 4gb limitation. – Cheeso Mar 05 '09 at 09:32
  • DotNetZip is a managed .NET library for ZIP files, that does ZIP64. It's free (gratis + libre), though donations are encouraged. ZIP64 is a solution to a different problem than the original poster asked about - the limitation of the BCL DeflateStream. – Cheeso Jun 24 '09 at 04:02
1

Look for libraries that support DEFLATE64 (not Zip64, that's an extension to the ZIP file format). Xceed Zip for .NET does support Deflate64, and I'm sure others do too.

Martin Plante
  • 4,553
  • 3
  • 33
  • 45
  • Um, no. deflate64 has nothing whatsoever to do with the size of the data. It only differs from deflate in providing a 64K sliding window instead of a 32K sliding window. – Mark Adler Mar 26 '14 at 04:01
0

DotNetZip does ZIP64 for .NET, and it is free. But Zip64 is not the same as Deflate64.

Cheeso
  • 189,189
  • 101
  • 473
  • 713
0

Although that documentation says the 4GB limitation is for both the DeflateStream and GZipStream, only GZipStream is limited because of the CRC32 checksum. If you do not need CRC32 then use DeflateStream.

  • Interesting. Couple questions. #1: Are you sure about this? #2. Why does the CRC32 limit the size of the data that can be compressed? #3. What is the source of your information? (are you familiar with the DeflateStream implementation, or what?) – Cheeso Jun 24 '09 at 04:09
  • Hi, 1. I am absolutely sure - I have tested it. 2. See Wikipedia: http://en.wikipedia.org/wiki/Cyclic_redundancy_check 3. I used .NET Reflector (http://www.red-gate.com/products/reflector/) to peek at the code. The 4GB limitation is completely ignored if CRC32 is not used. –  Aug 29 '09 at 17:04