Forwarding a stream in c# api

Question

In my service we need to get a zip file created by another service and return it.

This is my code (code has been simplified for the question):

[HttpGet("mediafiles/{id}")]
public async Task<IActionResult> DownloadMediaFiles(int id)
{
    var fileIds = _myProvider.GetFileIdsForEntityId(id); // result be like "1,2,3,4"

    using var httpClient = new HttpClient();
    var response = await httpClient.GetAsync($"http://file-service/bulk/{fileIds}");
    var stream = await response.Content.ReadAsStreamAsync();

    return File(stream, "application/octet-stream", "media_files.zip");
}

With the id I can gather the info I need to create the fileIds string and call the other service.

Here's the api on the other service (code has been simplified for the question):

[HttpGet("bulk/{idList}")]
public async Task<IActionResult> DownloadBulk(string idList)
{
    var ids = string.IsNullOrEmpty(idList) ? new List<int>() : idList.Split(',').Select(x => Convert.ToInt32(x));

    using var memoryStream = new MemoryStream();
    using (var archive = new ZipArchive(memoryStream, ZipArchiveMode.Create, true))
    {
        var index = archive.CreateEntry("hello.txt");
        using (var entryStream = index.Open())
        using (var streamWriter = new StreamWriter(entryStream))
        {
            streamWriter.Write("hello");
        }
    }

    var byteArray = memoryStream.ToArray();

    return File(byteArray, "application/octet-stream", "media_files.zip");
}

but when the client tries to open the zip we get

Exception has occurred. ArchiveException (FormatException: Could not find End of Central Directory Record)

I'm absolutely not confident about these two lines of the /mediafiles/{id}

var stream = await response.Content.ReadAsStreamAsync();

return File(stream, "application/octet-stream", "media_files.zip");

And probably the issue might be there.

I just need to forward back the file-service response, but I don't know why

Should you be returning FileStreamResult instead of File? – MikeJ Jun 09 '20 at 20:26 — MikeJ, Jun 09 '20 at 20:26

score 6 · Accepted Answer · answered Jun 09 '20 at 20:37

I believe the problem you're experiencing is that in DownloadMediaFiles(int id) you are using an HttpClient that gets disposed when leaving the function scope. The stream you created from the response therefore is closed and disposed of as well, before the response payload has finished writing its contents to the client. The client therefore receives an incomplete zip-file that you can't open. See here for reference.

In this answer there's a simple solution you could use, which is simply to read the response stream (the response stream from $"http://file-service/bulk/{fileIds}") into a byte array and then pass it to the response to the client:

using var httpClient = new HttpClient();
var response = await httpClient.GetAsync($"http://file-service/bulk/{fileIds}");
var byteArr = await response.Content.ReadAsByteArrayAsync();

return File(byteArr, "application/octet-stream", "media_files.zip");

You might realize that this means loading the whole file into memory, which can quickly become an issue if you plan on working with large files or even with medium sized files if the API is supposed to be used by a lot of clients simultaneously. Your web application would most likely run out of memory at some point.

Instead, I came upon this article which shows how you can return the contents of the stream from a request using an HttpClient. You should be able to stick with the first section of that article (all the ZIP-file and callback-based response stuff is unrelated).

To recap on that article all you need is something like this:

// Your ControllerClass.cs

private static HttpClient Client { get; } = new HttpClient();

[HttpGet("mediafiles/{id}")]
public async Task<IActionResult> DownloadMediaFiles(int id)
{
    var fileIds = _myProvider.GetFileIdsForEntityId(id); // result be like "1,2,3,4"
    var stream = await Client.GetStreamAsync($"http://file-service/bulk/{fileIds}");

    return File(stream, "application/octet-stream", "media_files.zip");
}

You'll notice, that the stream object is not disposed of here but ASP.Net Core does this for you as part of writing the response payload to the client. The Client which is stored in a static global variable is not disposed of either, which means you can reuse it between requests (it's usually recommended not to instantiate a new HttpClient everytime you need it). ASP.Net Core 2.1 and up has special support for dependency injecting the client for you through the IHttpClientFactory interface. I would suggest you do that instead of a static variable. Read here for the most basic usage of injecting the client factory.

Now you should be able to enjoy streaming the file contents directly from your "other service" without loading it into memory in your API web application.

Great response! I wish I could upvote it more... thank you! – Doc Jun 10 '20 at 16:40 — Doc, Jun 10 '20 at 16:40

Forwarding a stream in c# api

1 Answers1