Create ziparchive native c# out of memory memorystream

Question

I am using the build in feature to make a zip system.io.compression.ziparchive

https://msdn.microsoft.com/en-us/library/system.io.compression.ziparchive.getentry%28v=vs.110%29.aspx

and got:

An exception of type 'System.OutOfMemoryException' occurred in mscorlib.dll but was not handled in user code

I have read this: ZipArchive creates invalid ZIP file and this is some of my code in an *.ashx file (webforms):

Dictionary<string, string> csvs = new Dictionary<string, string>();
using (var memoryStream = new MemoryStream())
{
    using (var zip = new ZipArchive(memoryStream, ZipArchiveMode.Update, true))
    {
        // start loop for 2000 products
        zip.CreateEntryFromFile(prodImg, brandName + @"\" + dr["ProductPictureName"]);
        // per product CreateEntryFromFile * 4

        // more lines
        csvs[tmpName] = value + sb.ToString();
    }
}
memoryStream.Seek(0, SeekOrigin.Begin);
context.Response.BinaryWrite(memoryStream.ToArray());
context.Response.Flush();
context.Response.Close();
context.Response.End();

I have also read this https://stackoverflow.com/a/15869303/169714 and have no idea...

I have commented out some lines and get an out of memory when I append the output of a stringbuilder to a dictionary of type <string, string>

The funny thing is that when I used the DotNetZip nuget, it worked https://dotnetzip.codeplex.com/ But I prefer native framework rather then another nuget.

I have read this and can verify that the zip will not exceed 500mb http://bhrnjica.net/2012/07/22/with-net-4-5-10-years-memory-limit-of-2-gb-is-over/

ps seems to be related to having 8000 times CreateEntryFromFile tried to add CompressionLevel.NoCompression to reduce memory footprint, but did not solve my out of memory.

edit 2: I have reduced the 4 different images per product to just one for testing purposes and still have out of memory. Tried to move from CreateEntryFromFile to a more manual method...

//zip.CreateEntryFromFile(prodImg, brandName + @"\" + dr["ProductPictureName"], CompressionLevel.NoCompression);
ZipArchiveEntry zae = zip.CreateEntry(brandName + @"\" + dr["ProductPictureName"]);
using (var fileStream = File.OpenRead(prodImg))
{
    fileStream.CopyTo(zae.Open());
}

Both options do not work.

score -1 · Answer 1 · answered Nov 24 '15 at 14:20

-1

You need to avoid the MemoryStream, and try to write to the output stream as soon as possible. Maybe something like this can help.

using (var zip = new ZipArchive(context.Response.OutputStream, ZipArchiveMode.Update, true))
{
    // start loop for 2000 products
    zip.CreateEntryFromFile(prodImg, brandName + @"\" + dr["ProductPictureName"]);
    // per product CreateEntryFromFile * 4

    // more lines
    // csvs[tmpName] = value + sb.ToString(); // Also avoid the dictionary and the StringBuilder if you were using it just for debugging.
}

Hope that helps.

answered Nov 24 '15 at 14:20

Fede

3,928
1
20
28

You can't create it directly using Response.OutputStream as that doesn't support all the needed operations. – Anders Rune Jensen Jun 20 '17 at 12:35
@AndersRuneJensen can you elaborate into which needed operations are missing? I didn't test the code myself, as I stated "maybe something like this can help". The fact that you should avoid creating the zip in memory still remains. Maybe if OutputStream doesn't support all required operations, you can create an adapter stream that implements those operations, and flushes to the output stream ASAP. – Fede Jul 05 '17 at 17:39

Create ziparchive native c# out of memory memorystream

1 Answers1

Linked