Copy file from remote server to client browser via my server without writing the entire file to disk or memory

Question

This is the scenario.

There is a file on a remote file server (say I have a file hosted on DropBox)

I want to offer that file as a download on my web application (c# asp.net 4.0)

I want to hide the location 100% of the original file (I want it to appear to come from my sever).

I do not want to write this file to memory or to disk on my server.

I had assumed that I would want to use a stream to stream copy. example

Stream inputStream = response.GetResponseStream();
inputStream.CopyTo(Response.OutputStream, 4096);

inputStream.Flush();

Response.Flush();
Response.End();

This however copies the entire stream into memory before writing it out to the client browser. Any ideas would be awesome.

I need my server to basically just act as a proxy and shield the original file location

Thanks for any help.

you can't just obfuscate the url. There is no way to do this that would scale very well. — bumble_bee_tuna, Apr 05 '13 at 20:40
@bumble_bee_tuna, proxy servers scale perfectly fine - especially if stateless - just add as many servers as you need. — Alexei Levenkov, Apr 05 '13 at 20:43

user1260249 · Answer 1 · 2013-04-08T15:08:34.760

The following code is what I ended up with (the buffer size will change in production) This gets a file from a url, and starts streaming it chunk by chunk to a client through my server. The part that took me a bit to figure out was flushing the Response after each chunk was written.

HttpWebRequest request = (HttpWebRequest)WebRequest.Create(digitalAsset.FullFilePath);
            HttpWebResponse response = (HttpWebResponse)request.GetResponse();
            if (response.StatusCode == HttpStatusCode.OK)
            {
                Response.ClearHeaders();
                Response.ClearContent();
                Response.Clear();
                Response.ContentType = "application/octet-stream";
                Response.AddHeader("Content-Disposition", "attachment;filename=" +     digitalAsset.FileName);

                Stream inputStream = response.GetResponseStream();

                byte[] buffer = new byte[512];
                int read;
                while ((read = inputStream.Read(buffer, 0, buffer.Length)) > 0)
                {
                    Response.OutputStream.Write(buffer, 0, read);
                    Response.Flush();
                }
                Response.End();
            }

This can stream any size file through my server without waiting for my server to fist store it into memory or onto disk. The real advantage is that it uses very little memory as only the buffered chunk is stored. To the client the downloads begins instantly. (this is probably obvious to most but was very cool for this novice to see) so we are simultaneously downloading the file from one location and uploading it to another using the server as a sort of proxy.

score 1 · Accepted Answer · answered Apr 05 '13 at 20:42

1

While I don't see reasons why CopyTo will read whole stream in memory (as there is no intermediate stream anywhere), you can write the same Copy manually to be sure it behaves the way you want.

Consider using Async versions of read/write and if you can use async/awit from C# 4.0 to make code readable.

Old way: Asynchronous Stream Processing, new ways Asynchronous File I/O

answered Apr 05 '13 at 20:42

Alexei Levenkov

98,904
14
127
179

This was a great help. I am going to make a reply with the finished code in case someone else needs to do this. – user1260249 Apr 08 '13 at 14:57
1

CopyTo was not what was at fault. Well, kind of. It was the fact that the Response object was not getting flushed until the entire stream was written. I had to create the method from scratch in order to add flushing the Stream after each chunk. – user1260249 Apr 08 '13 at 15:11

Copy file from remote server to client browser via my server without writing the entire file to disk or memory

2 Answers2

Linked