3

How can I asynchronously take multiple existing streams (from the db), add them to a zip archive stream and return it in asp.net web api 2?

The key difference with the other "duplicate" question is how to do this in a streaming fashion without writing it to a temp file or buffer it completely in memory first.

AndyD
  • 5,252
  • 35
  • 32
  • What do you mean, asynchronously? – Alexander Jan 20 '14 at 14:17
  • 1
    [Create Zip File in Memory](http://stackoverflow.com/questions/17232414/creating-a-zip-archive-in-memory-using-system-io-compression) – Jonesopolis Jan 20 '14 at 14:18
  • 4
    without reading into memory first? What kind of magic are you thinking of using? – Joe Jan 20 '14 at 14:19
  • 2
    @Joe Stead: By streaming it from the db into the zip output stream. No magic required. – AndyD Jan 20 '14 at 14:24
  • @Alexander: see here http://www.hanselman.com/blog/TheMagicOfUsingAsynchronousMethodsInASPNET45PlusAnImportantGotcha.aspx – AndyD Jan 20 '14 at 14:25
  • Creating a zip file is a synchronous task. A download is a synchronous task. Request the file, prepare the file, serve the file. If the client is using a download manager, multiple concurrent requests can happen. Your server should support that. Hence, a ZIP file in memory is not a good choice imo, since it takes up the memory space. Why do you want it async? – Alexander Jan 20 '14 at 14:28
  • I want it async so it is performant and doesn't tie up worker threads (the whole reason why we have async controller actions) and I don't want it it in all read into memory first to reduce memory consumption and not have to introduce arbitrary size limits. – AndyD Jan 20 '14 at 14:30
  • 1
    @Jonesy: thanks for providing a good starting point. It's not async but at least you're being constructive. – AndyD Jan 20 '14 at 14:30
  • @Alexander: I don't need to support download managers or the ability to resume downloads. If I do, I would create the zip file on disk first to support that. – AndyD Jan 20 '14 at 14:33
  • Could someone explain why this question was downvoted? Thanks – AndyD Jan 20 '14 at 15:18
  • It wasn't me, though I still think that the async approach is not right here. The web is synchronous, and multiple files cannot be added at the same time to a ZIP. One after the other, then the Response. I don't see how async will improve, in the end you'll have to write code to sync the async. – Alexander Jan 20 '14 at 15:22
  • ok I understand your concern here. I can make the controller async as it does an async db call to get a sequential reader of streams, but the creating of the zip itself as that has to syncronous. I.e read/write one entry at a tie correct? – AndyD Jan 20 '14 at 15:33
  • Yes. And secondly, since you're preparing a response to a web request (aren't you?), the user has to wait for the result anyhow. – Alexander Jan 20 '14 at 15:43
  • 1
    The client should start reading from the response stream as soon as I do my first write to it. I don't want to write a whole zip file into a buffered memorystream before responding to the client. – AndyD Jan 20 '14 at 15:46
  • 1
    This question has been wrongfully marked as a duplicate: it asks for more than the other. I cannot add a new answer, but **Stephan Cleary has presented an ASP.NET Core alternative for `PushStreamContent`, which can write data to the output stream while zipping it on the fly**: https://blog.stephencleary.com/2016/11/streaming-zip-on-aspnet-core.html. (Note that the `WriteOnlyStreamWrapper` is no longer needed as of .NET Core 2.0.) – Timo May 03 '18 at 10:00
  • Looks to me like this is a duplicate of the other question. The other questioner wanted to do this. The issue here is whether the answers over there adequately cover the issue of streaming without the use of file system. I'm not sure that SE has a mechanism that adequately deals with this except perhaps allowing this question as a "near duplicate" in the hope of getting more specific answers. – Dominic Cronin May 03 '19 at 07:45

1 Answers1

0

It looks like you can't do this directly

Writing to ZipArchive using the HttpContext OutputStream

The http response stream needs to support seeking for a zip to be written directly to it which it doesn't. Will need to write to a temporary file by the looks of it.

Community
  • 1
  • 1
AndyD
  • 5,252
  • 35
  • 32
  • 1
    I have added [an answer](http://stackoverflow.com/a/21513194/41071) to that question that describes the issue and how to work around it. – svick Feb 02 '14 at 16:52
  • that answer answers most of my question. Can we now make it async to? – AndyD Feb 02 '14 at 23:18
  • I'm not sure if that's actually going to help you in any way, but the `Stream` returned by `ZipArchiveEntry.Open()` can be used asynchronously. – svick Feb 03 '14 at 00:21
  • I think I'll try to make my controller action async by reading from the db async and maybe leave the zip stuff sync. – AndyD Feb 04 '14 at 07:58