Zip within a zip opens to undocumented System.IO.Compression.SubReadStream

Question

I have a function I use for aggregating streams from a zip archive.

private void ExtractMiscellaneousFiles()
{
    foreach (var miscellaneousFileName in _fileData.MiscellaneousFileNames)
    {
        var fileEntry = _archive.GetEntry(miscellaneousFileName);
        if (fileEntry == null)
        {
            throw new ZipArchiveMissingFileException("Couldn't find " + miscellaneousFileName);
        }

        var stream = fileEntry.Open();
        OtherFileStreams.Add(miscellaneousFileName, (DeflateStream) stream);
    }
}

This works well in most cases. However, if I have a zip within a zip, I get an excpetion on casting the stream to a DeflateStream:

System.InvalidCastException: Unable to cast object of type 'System.IO.Compression.SubReadStream' to type 'System.IO.Compression.DeflateStream'.

I am unable to find Microsoft documentation for a SubReadStream. I would like my zip within a zip as a DeflateStream. Is this possible? If so how?

UPDATE

Still no success. I attempted @Sunshine's suggestion of copying the stream using the following code:

private void ExtractMiscellaneousFiles()
{
    _logger.Log("Extracting misc files...");

    foreach (var miscellaneousFileName in _fileData.MiscellaneousFileNames)
    {
        _logger.Log($"Opening misc file stream for {miscellaneousFileName}");

        var fileEntry = _archive.GetEntry(miscellaneousFileName);
        if (fileEntry == null)
        {
            throw new ZipArchiveMissingFileException("Couldn't find " + miscellaneousFileName);
        }

        var openStream = fileEntry.Open();
        var deflateStream = openStream;
        if (!(deflateStream is DeflateStream))
        {
            var memoryStream = new MemoryStream();
            deflateStream.CopyTo(memoryStream);
            memoryStream.Position = 0;
            deflateStream = new DeflateStream(memoryStream, CompressionLevel.NoCompression, true);
        }
        OtherFileStreams.Add(miscellaneousFileName, (DeflateStream)deflateStream);
    }
}

But I get a

System.NotSupportedException: Stream does not support reading.

I inspected deflateStream.CanRead and it is true.

I've discovered this happens not just on zips, but on files that are in the zip but are not compressed (because too small, for example). Surely there's a way to deal with this; surely someone has encountered this before. I'm opening a bounty on this question.

Here's the .NET source for SubReadStream, thanks to @Quantic.

Must be .NET Core: https://github.com/dotnet/corefx/blob/bffef76f6af208e2042a2f27bc081ee908bb390b/src/System.IO.Compression/src/System/IO/Compression/ZipCustomStreams.cs — Quantic, Oct 27 '16 at 19:40
Huh good find. Anything I can do with that? I noticed it's labeled as `internal`. — Scotty H, Oct 27 '16 at 19:44
Eh I'm not very knowledgeable here.. but, that [`Open()`](https://github.com/dotnet/corefx/blob/bffef76f6af208e2042a2f27bc081ee908bb390b/src/System.IO.Compression/src/System/IO/Compression/ZipArchiveEntry.cs) call appears to return a `Stream`, but the derived type is actually `SubReadStream` I guess? Anyways, because it's a `Stream`, maybe you just need to call the [`DeflateStream`](https://msdn.microsoft.com/en-us/library/hh137322(v=vs.110).aspx) constructor that accepts a stream, i.e., you can't cast it but I think you can just construct a new one. — Quantic, Oct 27 '16 at 19:50
@Quantic Good thought. Trying that, however, yielded a `System.ArgumentException: The base stream is not writeable.` Debugger inspection confirmed that `stream.CanWrite` is false; — Scotty H, Oct 27 '16 at 20:00
I think you should tag this with `.net-core` as these are core-specific functions. I may be wrong, but I feel like they are using an undocumented feature that doesn't grab the nested .zip as a file and instead it "unzips" it for you via the substream. But if true it seems they should still give a way for you to actually retrieve the .zip file itself instead of only allowing you to retrieve what's inside that zip as unzipped files. Either way, not having the right tag might be why someone who knows what's going on hasn't showed up yet. — Quantic, Oct 28 '16 at 19:15

Hans Passant · Accepted Answer · 2016-11-29T20:41:41.130

6

The return type of ZipArchiveEntry.Open() is Stream. An abstract type, in practice it can be a DeflateStream (you'd be happy), a SubReadStream (boo) or a WrappedStream (boo). Woe be you if they decide to improve the class some day and use a ZopfliStream (boo). The workaround is not good, you are trying to deflate data that is not compressed (boo).

Too many boos.

Only good solution is to change the type of your OtherFileStreams member. We can't see it, smells like a List<DeflateStream>. It needs to be a List<Stream>.

edited Nov 29 '16 at 20:41

answered Nov 29 '16 at 20:20

Hans Passant

922,412
146
1,693
2,536

1

Solved by using a `MemoryStream` instead of a `DeflateStream`. Thanks for the push to solve at a higher level. – Scotty H Dec 02 '16 at 19:48

score 1 · Answer 2 · answered Oct 28 '16 at 20:25

So it looks like the when storing a zip file inside another zip it doesn't deflate the zip but rather just inlines the content of the zip with the rest of the files with some information that these entries are part of a sub zip file. Which makes sense because applying compression to something that is already compressed is a waste of time.

This zip file is marked as CompressionMethodValues.Stored in the archive, which causes .NET to just return the original stream it read instead to wrapping it in a DeflateStream.

Source here: https://github.com/dotnet/corefx/blob/master/src/System.IO.Compression/src/System/IO/Compression/ZipArchiveEntry.cs#L670

You could pass the stream into a ZipArchive, if it's not a DeflateStream (if you are interested in the file inside)

var stream = entry.Open();
if (!(stream is DeflateStream))
{
    var subArchive = new ZipArchive(stream);
}

Or you can copy the stream to a FileStream (if you want to save it to disk)

var stream = entry.Open();
if (!(stream is DeflateStream))
{
    var fs = File.Create(Path.GetTempFileName());
    stream.CopyTo(fs);
    fs.Close();
}

Or copy to any stream you are interested in using.

Note: This is also how .NET 4.6 behaves

Zip within a zip opens to undocumented System.IO.Compression.SubReadStream

2 Answers2