Download file with WebClient or HttpClient?

Question

I am trying to download file from a URL and I have to choose between WebClient and HttpClient. I have referenced this article and several other articles on the internet. Everywhere, it is suggested to go for HttpClient due to its great async support and other .Net 4.5 privileges. But I am still not totally convinced and need more inputs.

I am using below code to download file from internet:

WebClient:

WebClient client = new WebClient();
client.DownloadFile(downloadUrl, filePath);

HttpClient:

using (HttpClient client = new HttpClient())
{        
    using (HttpResponseMessage response = await client.GetAsync(url))
    using (Stream streamToReadFrom = await response.Content.ReadAsStreamAsync())
    {
    }
}

From my perspective, I can see only one disadvantage in using WebClient, that would be the non async call, blocking the calling thread. But what if I am not worried about the blocking of thread or use client.DownloadFileAsync() to leverage the async support?

On the other hand, if I use HttpClient, ain't I loading every single byte of a file into memory and then writing it to a local file? If the file size is too large, won't memory overhead be expensive? Which could be avoided if we use WebClient, since it will directly write to local file and not consume system memory.

So, if performance is my utter priority, which approach should I use for download? I would like to be clarified if my above assumption is wrong, and I am open to alternate approach as well.

Answer 1

You can do it natively with .Net 4.5+. I tried doing it your way and then I just found a method in Intellisense that seemed to make sense.

https://learn.microsoft.com/en-us/dotnet/api/system.io.stream.copytoasync?view=netframework-4.7.2

uri = new Uri(generatePdfsRetrieveUrl + pdfGuid + ".pdf");
var response = await httpClient.GetAsync(uri);
using (var fs = new FileStream(
    HostingEnvironment.MapPath(string.Format("~/Downloads/{0}.pdf", pdfGuid)), 
    FileMode.CreateNew))
{
    await response.Content.CopyToAsync(fs);
}

Answer 2

Here is my approach.

If you are calling a WebApi to get a file, then from a controller method you can use HttpClient GET request and return file stream using FileStreamResult return type.

public async Task<ActionResult> GetAttachment(int FileID)
{
    UriBuilder uriBuilder = new UriBuilder();
    uriBuilder.Scheme = "https";
    uriBuilder.Host = "api.example.com";

    var Path = "/files/download";
    uriBuilder.Path = Path;
    using (HttpClient client = new HttpClient())
    {
        client.BaseAddress = new Uri(uriBuilder.ToString());
        client.DefaultRequestHeaders.Accept.Clear();
        client.DefaultRequestHeaders.Add("authorization", access_token); //if any
        client.DefaultRequestHeaders.Accept.Add(new System.Net.Http.Headers.MediaTypeWithQualityHeaderValue("application/json"));
        HttpResponseMessage response = await client.GetAsync(uriBuilder.ToString());

            if (response.IsSuccessStatusCode)
            {
                System.Net.Http.HttpContent content = response.Content;
                var contentStream = await content.ReadAsStreamAsync(); // get the actual content stream
                return File(contentStream, content_type, filename);
            }
            else
            {
                throw new FileNotFoundException();
            }
    }
}

Answer 3

To use HttpClient on my existing code that used WebClient, I wrote a small extension method to use it on the same way I used DownloadFileTaskAsync on my code.

using (var client = new System.Net.Http.HttpClient()) // WebClient
{
    var fileName = @"C:\temp\imgd.jpg";
    var uri = new Uri("https://yourwebsite.com/assets/banners/Default.jpg");

    await client.DownloadFileTaskAsync(uri, fileName);
}

To use it we can have this extension method:

public static class HttpClientUtils
{
    public static async Task DownloadFileTaskAsync(this HttpClient client, Uri uri, string FileName)
    {
        using (var s = await client.GetStreamAsync(uri))
        {
            using (var fs = new FileStream(FileName, FileMode.CreateNew))
            {
                await s.CopyToAsync(fs);
            }
        }
    }
}

Answer 4

Here’s one way to use it to download a URL and save it to a file: (I am using windows 7, therefore no WindowsRT available to me, so I’m also using System.IO.)

public static class WebUtils
{
    private static Lazy<IWebProxy> proxy = new Lazy<IWebProxy>(() => string.IsNullOrEmpty(Settings.Default.WebProxyAddress) ? null : new WebProxy { Address = new Uri(Settings.Default.WebProxyAddress), UseDefaultCredentials = true });

    public static IWebProxy Proxy
    {
        get { return WebUtils.proxy.Value; }
    }

    public static Task DownloadAsync(string requestUri, string filename)
    {
        if (requestUri == null)
            throw new ArgumentNullException(“requestUri”);

        return DownloadAsync(new Uri(requestUri), filename);
    }

    public static async Task DownloadAsync(Uri requestUri, string filename)
    {
        if (filename == null)
            throw new ArgumentNullException("filename");

        if (Proxy != null)
            WebRequest.DefaultWebProxy = Proxy;

        using (var httpClient = new HttpClient())
        {
            using (var request = new HttpRequestMessage(HttpMethod.Get, requestUri))
            {
                using (Stream contentStream = await (await httpClient.SendAsync(request)).Content.ReadAsStreamAsync(), stream = new FileStream(filename, FileMode.Create, FileAccess.Write, FileShare.None, Constants.LargeBufferSize, true))
                {
                    await contentStream.CopyToAsync(stream);
                }
            }
        }
    }
} 

Note that code is saving the address of the proxy server I use (at work) in a setting, and using that if such setting is specified. Otherwise, it should tell you all you need to know regarding using the HttpClient beta to download and save a file.

Answer 5

For code being called repeatedly, you do not want to put HttpClient in a using block (it will leave hanging ports open)

For downloading a file with HttpClient, I found this extension method which seemed like a good and reliable solution to me:

public static class HttpContentExtensions
{
    public static Task ReadAsFileAsync(this HttpContent content, string filename, bool overwrite)
    {
        string pathname = Path.GetFullPath(filename);
        if (!overwrite && File.Exists(filename))
        {
            throw new InvalidOperationException(string.Format("File {0} already exists.", pathname));
        }

        FileStream fileStream = null;
        try
        {
            fileStream = new FileStream(pathname, FileMode.Create, FileAccess.Write, FileShare.None);
            return content.CopyToAsync(fileStream).ContinueWith(
                (copyTask) =>
                {
                    fileStream.Close();
                });
        }
        catch
        {
            if (fileStream != null)
            {
                fileStream.Close();
            }

            throw;
        }
    }
}

Answer 6

If you want (or have) to do this synchronously, but using the nice HttpClient class, then there's this simple approach:

string requestString = @"https://example.com/path/file.pdf";

var GetTask = httpClient.GetAsync(requestString);
GetTask.Wait(WebCommsTimeout); // WebCommsTimeout is in milliseconds

if (!GetTask.Result.IsSuccessStatusCode)
{
    // write an error
    return;
}
                    
using (var fs = new FileStream(@"c:\path\file.pdf", FileMode.CreateNew))
{
    var ResponseTask = GetTask.Result.Content.CopyToAsync(fs);
    ResponseTask.Wait(WebCommsTimeout);
}

Answer 7

My approach is very simple. Using FileStream you can store it in the local folder, or return it from API using FileStreamResult. Example for store into local folder:

private async Task SaveDataIntoLocalFolder(string url,string fileName)
{
    using (var client = new HttpClient())
    {
        var response = await client.GetAsync(url);
        if (response.IsSuccessStatusCode)
        {
           var stream = await response.Content.ReadAsStreamAsync();
           var fileInfo = new FileInfo(fileName);
           using (var fileStream = fileInfo.OpenWrite())
           {
              await stream.CopyToAsync(fileStream);
           }
        }
        else
        {
          throw new Exception("File not found");
        }
    }     
}

Answer 8

This is a simple demo UWP application for downloading an image file. Just paste the image URL link and press the download button. You can identify the file type and change the fileName to download the desired file.

MainPage.xaml

<Page
    x:Class="HttpDownloader.MainPage"
    xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation"
    xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml"
    xmlns:local="using:HttpDownloader"
    xmlns:d="http://schemas.microsoft.com/expression/blend/2008"
    xmlns:mc="http://schemas.openxmlformats.org/markup-compatibility/2006"
    mc:Ignorable="d"
    Background="{ThemeResource ApplicationPageBackgroundThemeBrush}">

    <Grid>
        <StackPanel>
            <TextBox x:Name="uriInput"
                     Header="URI:" PlaceholderText="Please provide an uri"
                     Width="300"
                     HorizontalAlignment="Center"/>
            <Button Content="Dowload"
                    HorizontalAlignment="Center"
                    Click="Button_Click"/>
        </StackPanel>
    </Grid>
</Page>

MainPage.xaml.xs

using System;
using System.Collections.Generic;
using System.IO;
using System.Linq;
using System.Net.Http;
using System.Runtime.InteropServices.WindowsRuntime;
using System.Threading.Tasks;
using Windows.Foundation;
using Windows.Foundation.Collections;
using Windows.UI.Xaml;
using Windows.UI.Xaml.Controls;
using Windows.UI.Xaml.Controls.Primitives;
using Windows.UI.Xaml.Data;
using Windows.UI.Xaml.Input;
using Windows.UI.Xaml.Media;
using Windows.UI.Xaml.Navigation;
using System.Net.Http;
using System.Net;
using Windows.Storage.Streams;
using Windows.Storage.Pickers;
using Windows.Storage;
using Windows.Graphics.Imaging;
using System.Threading;

// The Blank Page item template is documented at https://go.microsoft.com/fwlink/?LinkId=402352&clcid=0x409

namespace HttpDownloader
{
    /// <summary>
    /// An empty page that can be used on its own or navigated to within a Frame.
    /// </summary>
    public sealed partial class MainPage : Page
    {
        public MainPage()
        {
            this.InitializeComponent();
        }

        private async void Button_Click(object sender, RoutedEventArgs e)
        {
            HttpClient client = new HttpClient();
            string imageUrl = uriInput.Text;
            try
            {
                using (var cancellationTokenSource = new CancellationTokenSource(50000))
                {
                    var uri = new Uri(WebUtility.HtmlDecode(imageUrl));
                    using (var response = await client.GetAsync(uri, cancellationTokenSource.Token))
                    {
                        response.EnsureSuccessStatusCode();
                        var mediaType = response.Content.Headers.ContentType.MediaType;
                        string fileName = DateTime.Now.ToString("yyyyMMddhhmmss");
                        if (mediaType.IndexOf("jpg", StringComparison.OrdinalIgnoreCase) >= 0
                            || mediaType.IndexOf("jpeg", StringComparison.OrdinalIgnoreCase) >= 0)
                        {
                            fileName += ".jpg";
                        }
                        else if (mediaType.IndexOf("png", StringComparison.OrdinalIgnoreCase) >= 0)
                        {
                            fileName += ".png";
                        }
                        else if (mediaType.IndexOf("gif", StringComparison.OrdinalIgnoreCase) >= 0)
                        {
                            fileName += ".gif";
                        }
                        else if (mediaType.IndexOf("bmp", StringComparison.OrdinalIgnoreCase) >= 0)
                        {
                            fileName += ".bmp";
                        }
                        else
                        {
                            fileName += ".png";
                        }

                        // Get the app's local folder.
                        StorageFolder localFolder = Windows.Storage.ApplicationData.Current.LocalFolder;

                        // Create a new subfolder in the current folder.
                        // Replace the folder if already exists.
                        string desiredName = "Images";
                        StorageFolder newFolder = await localFolder.CreateFolderAsync(desiredName, CreationCollisionOption.ReplaceExisting);
                        StorageFile newFile = await newFolder.CreateFileAsync(fileName, CreationCollisionOption.ReplaceExisting);

                        using (Stream streamStream = await response.Content.ReadAsStreamAsync())
                        {
                            using (Stream streamToWriteTo = File.Open(newFile.Path, FileMode.Create))
                            {
                                await streamStream.CopyToAsync(streamToWriteTo);
                            }
                        }
                    }
                }
            }
            catch (Exception ex)
            {
                Console.WriteLine("Exception occur");
                Console.WriteLine(ex.ToString());
            }
        }
    }
}

You will find the image in this folder.

Users/[current user name]/AppData/Local/Packages/[Application package name]/LocalState/Images

Answer 9

   HttpClient _client=new HttpClient();
   byte[] buffer = null;
   try
   {       
      HttpResponseMessage task = await _client.GetAsync("https://**FILE_URL**");
      Stream task2 = await task.Content.ReadAsStreamAsync();
      using (MemoryStream ms = new MemoryStream())
      {
        await task2.CopyToAsync(ms);
        buffer = ms.ToArray();
      }
      File.WriteAllBytes("C:/**PATH_TO_SAVE**", buffer);  
   }
   catch
   {

   }