WebClient is very slow

Question

I have problem with Webclient.

It is very slow. It takes about 3-5 seconds to downloadString from one website. I don't have any network problems.

This is my Modifed WebClient.

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Net;

namespace StatusChecker
{
    class WebClientEx: WebClient
    {
        public CookieContainer CookieContainer { get; private set; }

        public WebClientEx()
        {
            CookieContainer = new CookieContainer();

            ServicePointManager.Expect100Continue = false;
            Encoding = System.Text.Encoding.UTF8;

            WebRequest.DefaultWebProxy = null;
            Proxy = null;
        }

        public void ClearCookies()
        {
            CookieContainer = new CookieContainer();
        }

        protected override WebRequest GetWebRequest(Uri address)
        {

            var request = base.GetWebRequest(address);
            if (request is HttpWebRequest)
            {
                (request as HttpWebRequest).CookieContainer = CookieContainer;
            }
            return request;
        }
    }
}

UPDATE: In wireshark I saw that single DownladString is sending and receiving few thousands packets.

WebClient isn't especially slow, the problem must be somewhere else... Perhaps the website you're downloading from is slow? — Thomas Levesque, Aug 08 '11 at 21:49
You should try using a traffic logger like WireShark to determine whether the delay is indeed between single request and response frames (caused by network/server slowdown) or is happening because the client code is misbehaving and sending out too many requests or some such logic error. — D. A. Terre, Aug 08 '11 at 21:53
Also execute multiple downloads in a loop to 'warm up' the process (eliminate loaded of assemblies for example). — Philipp Schmid, Aug 08 '11 at 21:57
UPDATE: In wireshark I saw that single DownladString is sending and reciening few thousands packets. Please help me. — Hooch, Aug 08 '11 at 22:36
Sending and receiving so many packets could just be due to streaming and isn't in itself an indication that anything is wrong. — Quintin Robinson, Aug 08 '11 at 22:38
If the content you're downloading is really large, this is what you should *expect*. If you manually download the URL, how big is it? — Jacob, Aug 08 '11 at 22:39
Painfully slow!!!!! I've did some tests using the same webserver, same page, and got greatest speeds with: wget, curl, delphi test program, python test program, C# test program. (Yes, in this order. C# WebClient just sucks, and yes, did the proxy trick) — Filipe YaBa Polido, Oct 30 '14 at 21:36

foxy · Accepted Answer · 2012-08-30T10:05:40.770

31

There may be two issues at hand here (that I've also noticed in my own programs previously):

The first request takes an abnormally long time: This occurs because WebRequest by default detects and loads proxy settings the first time it starts, which can take quite a while. To stop this, simply set the proxy property (WebRequest.Proxy) to null and it'll bypass the check (provided you can directly access the internet)
You can't download more than 2 items at once: By default, you can only have 2 simultaneous HTTP connections open. To change this, set ServicePointManager.DefaultConnectionLimit to something larger. I usually set this to int.MaxValue (just make sure you don't spam the host with 1,000,000 connections).

edited Aug 30 '12 at 10:05

answered Aug 09 '11 at 00:52

foxy

7,599
2
30
34

1

This is not correct answer in my case. You can see that I allready set proxy to null. And I'm not downloadnig 2 items at the same time. As also added to my post that in Wireshark I can see that my app sends and recives thousands packets per one download string. String is simple website like google.com. – Hooch Aug 09 '11 at 07:52
If it is not abnormally slow (meaning connections have about the same speed with keepalives turned off), then it is working as it should. It takes time to execute the handshake, make the request, wait for the server to create a response, then to read the response. – foxy Aug 09 '11 at 08:13
Can you tell me little bit more about those keep alives? And why my app sends thousands of packets? – Hooch Aug 09 '11 at 09:18
A [packet](http://en.wikipedia.org/wiki/Network_packet) represents a piece of data. A file or HTML page generally consists of a large file, which is too large to transfer all in one go due to the design of the internet protocol. For that reason, we need to split a large file (pretty much anything over 1kB) into many parts to send them as a stream. Each part is a packet. What you're seeing is completely normal. Keep alives speed up reconnection, keeping a connection (to a particular server) alive, so that we don't have to handshake on every new request (to that particular server). – foxy Aug 09 '11 at 09:29
I'm connecting to only one server. How to keep alive? – Hooch Aug 09 '11 at 11:36
According to the MSDN documentation for `ServicePointManager.MaxServicePoints`, the default maximum is unlimited? – McKay Aug 29 '12 at 13:47
@Hooch Better late than never, but if you use a `WebClient` and keep the object alive throughout multiple requests, it'll do that automatically. – Aidiakapi Jun 13 '13 at 21:27
@Hooch Then it'll have automatically kept the connection alive unless you specified it not to :) – Aidiakapi Jun 16 '13 at 12:52

score 2 · Answer 2 · answered Oct 17 '17 at 06:43

There are a few options if it is related to the initial proxy settings being checked:

Disable the automatic proxy detection settings in Internet Explorer
Set the proxy to null:

WebClient.Proxy = null
On application startup set the default webproxy to null:

WebRequest.DefaultWebProxy = null;

In older .NET code instead of setting to null, you used to write this (but null is now preferred):

webclient.Proxy = GlobalProxySelection.GetEmptyWebProxy();

score 0 · Answer 3 · edited May 23 '17 at 10:29

0

Maybe it will help somebody. Some web services support compression (gzip or other). So you can add Accept-Encoding header for your requests and then enable automatic decompression for web client instance. Chrome works in that way.

edited May 23 '17 at 10:29

Community

1
1

answered Oct 16 '15 at 10:52

Dmitriy Dokshin

710
5
25

WebClient is very slow

3 Answers3

Linked