4

I have a code that get all files on a directory, compresses each one and creates a .zip file. I'm using the .NET Framework ZipArchive class on the System.IO.Compression namespace and the extension method CreateEntryFromFile. This is working well except when processing large files (aproximately 1GB and up), there it throws a System.IO.Stream Exception "Stream too large".

On the extension method reference on MSDN it states that:

When ZipArchiveMode.Update is present, the size limit of an entry is limited to Int32.MaxValue. This limit is because update mode uses a MemoryStream internally to allow the seeking required when updating an archive, and MemoryStream has a maximum equal to the size of an int.

So this explains the exception I get, but provides no further way of how to overcome this limitation. How can I allow large file proccesing?

Here is my code, its part of a class, just in case, the GetDatabaseBackupFiles() and GetDatabaseCompressedBackupFiles() functions returns a list of FileInfo objects that I iterate:

public void CompressBackupFiles()
{
    var originalFiles = GetDatabaseBackupFiles();
    var compressedFiles = GetDatabaseCompressedBackupFiles();
    var pendingFiles = originalFiles.Where(c => compressedFiles.All(d => Path.GetFileName(d.Name) != Path.GetFileName(c.Name)));
    foreach (var file in pendingFiles)
    {
        var zipPath = Path.Combine(_options.ZippedBackupFilesBasePath, Path.GetFileNameWithoutExtension(file.Name) + ".zip");
        using (ZipArchive archive = ZipFile.Open(zipPath, ZipArchiveMode.Update))
        {
             archive.CreateEntryFromFile(file.FullName, Path.GetFileName(file.Name));
        }
    }
    DeleteFiles(originalFiles);
}
jazb
  • 5,498
  • 6
  • 37
  • 44
David Ortega
  • 915
  • 9
  • 25
  • Are you creating a new zip file, or updating an existing zip file? When you are creating, why don't you use the create mode? When you are updating a existing zip file, you need to rework the process to use create... – Julo Oct 25 '18 at 02:40
  • @Julo I'm creating a new file for each existing file on the directory as presented on the samples on MSDN. So far it does work though. – David Ortega Oct 25 '18 at 02:42
  • Then use `ZipArchiveMode.Create` – Julo Oct 25 '18 at 02:43
  • @Julo that worked! Could you post it as an answer so I can mark this question solved? – David Ortega Oct 25 '18 at 03:03

1 Answers1

3

When you are only creating a zip file, replace the ZipArchiveMode.Update with ZipArchiveMode.Create.

The update mode is meant for cases, when you need delete files from an existing archive, or add new files to existing archive.

In the update mode the whole zip file is loaded into memory and it consumes a lot of memory for big files. Therefore this mode should be avoided when possible.

Julo
  • 1,102
  • 1
  • 11
  • 19