WebClient is opening a new connection each time I download a file and all of them stay established

Question

I'm having a problem finding out how to close the connection made by WebClient. I create a new WebClient object and then call DownloadFile method a lot of times, however, it always creates a new connection for each call and those connections stay open (Established state), I can see in TCPView all the established connections.

What bugs me even more if when I dispose the Webclient, they stay established...? How to force the connection to be closed after the download is done?

I already tried to derive WebClient and set keep alive manually to false, my app config allow enough connections as well.

<connectionManagement>
  <add address="*" maxconnection="1000"/>
</connectionManagement>

Answer 1

Short answer: you shouldn't need to close the connections manually. They are managed for you behind the scenes.

HTTP/1.1 connections are not closed as soon as the request completes in order that multiple requests to the same server are handled in a more timely and efficient manner (such as a web browser requesting multiple files from a single site). You shouldn't have to worry about this or close them down manually, as they'll timeout after a while. Is it causing an error?

If it is an issue, you could try inheriting from WebClient and overriding the GetWebRequest method to manually set KeepAlive, e.g.:

public class NoKeepAlivesWebClient : WebClient
{
    protected override WebRequest GetWebRequest(Uri address)
    {
        var request = base.GetWebRequest(address);
        if (request is HttpWebRequest)
        {
            ((HttpWebRequest)request).KeepAlive = false;
        }

        return request;
    }
}

I'd also always suggest using the using pattern for WebClient:

using (var client = new NoKeepAlivesWebClient())
{
    // Some code
}

Finally, here's some RFC information about persistent connections in HTTP/1.1:

http://www.w3.org/Protocols/rfc2616/rfc2616-sec8.html

and a friendlier Wikipedia entry:

http://en.wikipedia.org/wiki/HTTP_persistent_connection

Edit:

Apologies. I see from your edited question that you've already tried something like the above, without success.

I was unable to reproduce your issue, however. I wrote a small program using the NoKeepAlivesWebClient and it successfully closed connections after they were used, according to TCPView.

static void Main(string[] args)
{
    // Random test URLs
    var urls = new List<string> {
        "http://msdn.microsoft.com/en-us/library/tt0f69eh.aspx",
        "http://msdn.microsoft.com/en-us/library/system.net.webclient.allowreadstreambuffering.aspx",
        "http://msdn.microsoft.com/en-us/library/system.net.webclient.allowwritestreambuffering.aspx",
        "http://msdn.microsoft.com/en-us/library/system.net.webclient.baseaddress.aspx",
        "http://msdn.microsoft.com/en-us/library/system.net.webclient.cachepolicy.aspx",
        "http://msdn.microsoft.com/en-us/library/system.net.webclient.credentials.aspx",
        "https://www.youtube.com/",
        "https://www.youtube.com/feed/UClTpDNIOtgfRkyT-AFGNWVw",
        "https://www.youtube.com/feed/UCj_UmpoD8Ph_EcyN_xEXrUQ",
        "https://www.youtube.com/channel/UCn-K7GIs62ENvdQe6ZZk9-w" };

    using (var client = new NoKeepAlivesWebClient())
    {
        // Save each URL to a Temp file
        foreach (var url in urls)
        {
            client.DownloadFile(new Uri(url), System.IO.Path.GetTempFileName());
            Console.WriteLine("Downloaded: " + url);
        }
    }
}

There's another SO question here about the same issue:

C# get rid of Connection header in WebClient