0

I do realize that there are different wat to check the existence of a website in C# and other languages. However, I have come to understand that some of them don't work as good. For example, this code:

        WebClient client = new WebClient();
        try
        {
            string response = client.DownloadString(url);
            return true;  // Successful -> the website does exist.
        }
        catch
        {
            return false; // Unsuccessful -> the website doesn't exist.
        }

doesn't detect www.bb.com, although this website clearly does exist. What code should I use to be absolutely on the safe side here?

xiselen
  • 11
  • 1
  • 6
    "*URL exists or not?*" urls exist when you write them as text. However if you want to check if a remote resources exists you have to prod it. You could ping the domain, however it will likely be unsatisfactory, or you could send a head request, which might work ***most of the time***. – TheGeneral Sep 29 '20 at 04:53
  • 4
    To be honest `URL exists` is heaps too vague to define. e.g. imagine on a corporate network you may have intranet resources that are using private DNS, and are thus not even resolvable publicly. At the same time, corporate network may block public internet access preventing you from accessing it. Lastly, as pointed out by @MichaelRandall the resource may actually be unavailable temporarily, or you may have an ISP or other network issue (e.g. SSL/TLS cipher negotiation, time skew etc. etc.). The attempt to make statements as to website availability from the client machine is futile. – zaitsman Sep 29 '20 at 04:59

1 Answers1

0

Given that:

  1. The site you linked expects one of the main browsers to be set in the User-Agent string
  2. HttpClient is more recommended and newer than WebClient

you could do it like this:

    using System;
    using System.Net.Http;
    using System.Threading.Tasks;

    namespace resourcechecker
    {
        class Program
        {
            static HttpClient client = new HttpClient();

            public async static Task<bool> ResourceExists(HttpClient client, string url)
            {
                HttpResponseMessage response;
                response = await client.GetAsync(url);
                if (response.IsSuccessStatusCode)
                {
                    return true;
                }
                else
                {
                    return false;
                }
            }


            static void Main(string[] args)
            {
                client.DefaultRequestHeaders.Add("User-Agent", "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:64.0) Gecko/20100101 Firefox/64.0");
                string url = "http://www.bb.com/";
                Console.WriteLine($"It is {ResourceExists(client, url).Result} that {url} exists");
            }
        }
    }

However, this could still give the wrong answer if, for example, the IP address of the testing computer is blocked from accessing that web server

trajekolus
  • 535
  • 6
  • 16