How do I transmit a large, multi-gig file in ServiceStack?

Question

I'm using ServiceStack as my web framework and am trying to send a 3 gig file across the pipe. Below is the code I'm trying to use to send the file. It works for small files, but when I try to send large files the application stalls and never actually sends the files.

    public IHttpResult Any(TilesRequest request)
    {
        var filename = request.Name.ToLower();
        var filePath = HttpContext.Current.Server.MapPath(ConfigLookups.TilesDirectory + filename);
        if (File.Exists(filePath))
        {
            Response.AddHeader("content-disposition", "attachment;filename=" + filename);
            var stream = File.OpenRead(filePath);
            return new HttpResult(stream, "application/x-sqlite3");
        }
        else
        {
            return new HttpError(System.Net.HttpStatusCode.NotFound, "File not found");
        }
    }

I also tried doing:

Response.TransmitFile(filePath);

but then I get an error that says:

The size parameter must be between zero and the maximum Int32 value.

Any ideas on how I should properly be sending such a large file?

Answer 1

Response.TransmitFile in ServiceStack just delegates to ASP.NET's TransmitFile which is the API for efficiently sending large files - it uses an efficient Win32 API underneath which from the error description, looks like it's limited to 2 GB (2^32) in size. It's not clear if this is limitation is lifted in a 64bit OS.

Other limitations are imposed by ASP.NET (i.e. ServiceStack doesn't add any limitations itself). This previous thread shows a few tips you can try to send large files with ASP.NET.

This also applies to ServiceStack since it's just a wrapper around ASP.NET's IHttpHandler. You can get the underlying ASP.NET Response with:

var aspResponse = base.Response.OriginalResponse as HttpResponseBase;

The built-in ASP.NET httpRuntime request limits would also need adjusting.

You also want to disable buffering to avoid the built-in 2^32 size limit. ASP.NET does this by default when compression is enabled so you also want to disable this:

<system.webServer>
   <urlCompression doStaticCompression="true" doDynamicCompression="false" />
</system.webServer>

But generally if you exceed the 2^32 limit you're likely going to have issues trying to sending this much data at once in ASP.NET no matter what you do.

Partial Content Responses

The solution I'd be looking at for streaming large files instead is to use ServiceStack's built-in Partial Content Support. Which instead of streaming a large file in 1 request, change the HttpClient to fetch the file in chunks over multiple requests.

A Physical File

return new HttpResult(new FileInfo(filePath), request.MimeType); 

A Memory Stream

return new HttpResult(ms, "audio/mpeg");

Here are some examples of using HttpUtils to fetch Partial Content from ServiceStack Services, e.g:

var url = "{0}/{1}".Fmt(BaseUrl, new TilesRequest { ... }.ToUrl());
var partialBytes = url.GetBytesFromUrl(
    requestFilter: httpReq => httpReq.AddRange(Offset, ChunkSize),
    responseFilter: httpRes => {
        "Content-Length header {0}".Print(
           httpRes.Headers["Content-Length"]);
    });

You would need to do this in a loop to stitch up all the partial byte responses together.