0

I am developing a simple method for downloading html webpages :

        public string DownloadUsingWebRequest(string url)
    {
        HttpWebRequest request = (HttpWebRequest)WebRequest.Create(url);

        request.Method = "GET";
        request.UserAgent = "Mozilla / 5.0(Windows NT 10.0; Win64; x64) AppleWebKit / 537.36(KHTML, like Gecko) Chrome / 70.0.3538.77 Safari / 537.36";
        request.ServicePoint.Expect100Continue = true;

        ServicePointManager.SecurityProtocol |= SecurityProtocolType.Tls12 |
                                              SecurityProtocolType.Tls11;

        ServicePointManager.ServerCertificateValidationCallback =
            (a, b, c, d) => true;

        using (var response = (HttpWebResponse)(request.GetResponse()))
        {
            var code = response.StatusCode;
            if (code != HttpStatusCode.OK) return "";
            using (var sr = new StreamReader(response.GetResponseStream(), Encoding.UTF8))
            {
                return sr.ReadToEnd();
            }
        }
    }

However, When I run this snippet and pass a https url, the following exception is thrown :

System.Net.WebException: 'The request was aborted: Could not create SSL/TLS secure channel.'

I have looked into this link but didn't help much.

Note : The website that I am using as a test uses TLS 1.2.

Tarek Mohamed
  • 81
  • 1
  • 8
  • Try moving the `ServicePointManager` stuff to the top of the method. – rfmodulator Mar 19 '20 at 23:01
  • If you're on Windows 10, you don't need `SecurityProtocolType.Tls12`, the System default is already that. If you're on Windows 7 or Windows Server 2008, if - after you have moved `ServicePointManager.SecurityProtocol` **=** `SecurityProtocolType.Tls12` before the point where you create the connection - you have the same result, it's because your System doesn't have the `TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256` cipher suite. A growing number of Sites uses this cipher (and this one only - maybe its companion of the same suite). So, no luck (unless you bring in this cipher suite - and manage it). – Jimi Mar 20 '20 at 02:21
  • 1
    @rfmodulator I tried moving the `ServicePointManager` calls to the start of the method, same exception is thrown – Tarek Mohamed Mar 20 '20 at 09:41
  • @Jimi I have done some research on my test website and it actually uses the mentioned cipher suite, does that mean that there is no way I can read it's html text ? – Tarek Mohamed Mar 20 '20 at 09:46
  • You can try with a headless WebBrowser. – Jimi Mar 20 '20 at 10:27
  • @TarekMohamed Thanks for trying, I was curious because I've never tested the order. Since they're process-wide settings, I set them at startup. Is there a public URL that throws the same exception with your code, something you could share? – rfmodulator Mar 20 '20 at 16:20

0 Answers0