0

I have a Windows Service will be reading from local disk (video files) and post them to remote service via API.

Video files over 2gb size and I need to transfer them to another location through HttpClient/POST request.

There is no limitation on API call, so even if file is 10gb I can still post file to destination.

Right now I am reading entire video file as byte[] fileContent and pass it to function as

ByteArrayContent contentBody = new ByteArrayContent(fileContent);

It works for now, but since this is not scalable. (If multiple files will be transferred at the same time, it can fill up memory) I am seeking for a solution that transfer will happen in chunks.

Question: Can I read big files in buffer and transfer over HTTP as I am reading from local disk?

Teoman shipahi
  • 47,454
  • 15
  • 134
  • 158
  • Is this a .NET Web API or some other API? And are you able to make modifications on the receiving end? – Shaun Apr 16 '15 at 21:43
  • There's not enough information here. What are your requirements? is thisa server to client? Are you using your own software? Web Browser? – Ryan Ternier Apr 16 '15 at 21:44
  • @Shaun I updated question for clarification. This will be a Windows Service will be reading from local disk (video files) and post them to remote service via http post request. – Teoman shipahi Apr 16 '15 at 21:46
  • Have you had a look at this SO question? http://stackoverflow.com/questions/566462/upload-files-with-httpwebrequest-multipart-form-data - I think the answers there may contain what you're looking for, but there is some dependency on what the receiving end will accept. – Shaun Apr 16 '15 at 21:48

1 Answers1

3

You can use the PostAsync(Uri, HttpContent) method of HttpClient. In order to stream the contents of your local file, use the StreamContent subclass of HttpContent and supply a file reader stream. A brief example:

async Task PostBigFileAsync(Uri uri, string filename)
{
  using (var fileStream = File.OpenRead(filename))
  {
    var client = new HttpClient();
    var response = await client.PostAsync(uri, new StreamContent(fileStream));
  }
}
Dan Hermann
  • 1,107
  • 1
  • 13
  • 27
  • 1
    looks like I am still loading whole file into memory. I am looking for a solution that, it will read from file with buffer and post it as it goes. – Teoman shipahi Apr 17 '15 at 03:31
  • 1
    @Teomanshipahi Can you post the code you are using? I have used the snippet above to POST files that are multiple hundreds of megabytes and it doesn't use more than 2 or 3MB of RAM. The creation of the FileStream does not read the file in its entirety. Only at the point of the call to client.PostAsync is the file read and then only in buffered chunks that are immediately streamed out through the HttpClient. – Dan Hermann Apr 17 '15 at 12:13