Does CopyTo store the whole thing in memory?

Question

I have the following code snippet, which is designed to add files to a .zip file, while at the same time calculating their sha1 checksum.

However, it's running out of memory on large files.

Which part of it is causing the whole file to be in memory? Surely this should all be just streamed?

using (ZipArchive archive = ZipFile.Open(buildFile, ZipArchiveMode.Update))
{
    foreach (var fileName in nameList)
    {
        ZipArchiveEntry entry = archive.CreateEntry(source.filename);
        using (Stream zipData = entry.Open())
        using (SHA1Managed shaForFile = new SHA1Managed())
        using (Stream sourceFileStream = File.OpenRead(fileName))
        using (Stream sourceData = new CryptoStream(sourceFileStream, shaForFile, CryptoStreamMode.Read))
        {
            sourceData.CopyTo(zipData);
            print fileName + ':' + shaForFile.Hash;
        }
    }
}

@Yuval [no, it isn't](http://stackoverflow.com/questions/1933742/how-is-the-stream-copytostream-method-implemented-in-net-4). — CodeCaster, Jun 04 '15 at 11:39
@CodeCaster It reads in chunks, but eventually the entire file is in the `ZipArchive`. — Yuval Itzchakov, Jun 04 '15 at 11:42
Yeah so OP needs to now and then flush the zip file to disk. — CodeCaster, Jun 04 '15 at 11:44
It is not CopyTo() and not Sha1Managed, they use small buffers. The problem is ZipArchiveMode.Update, that can require significant alterations to the file on disk. It can only ever directly stream to disk when you use ZipArchiveMode.Create. I'd guess the only real workaround is to first extract the existing content so you can create the entire zip archive from scratch. — Hans Passant, Jun 04 '15 at 11:47
Thank you , that was it. Fortunately, I can use create mode here. Would you like to put this as an answer, then I can accept it? — xorsyst, Jun 04 '15 at 11:59

score 1 · Accepted Answer · answered Jun 05 '15 at 11:17

1

(Copied from a comment - as this answers the question)

The problem is ZipArchiveMode.Update, that can require significant alterations to the file on disk. It can only ever directly stream to disk when you use ZipArchiveMode.Create

answered Jun 05 '15 at 11:17

xorsyst

7,897
5
39
58

Does CopyTo store the whole thing in memory?

1 Answers1