How to check if it is 404 error page and do something using HtmlAgilityPack

Question

I'm trying to get multiple data from different urls using HtmlAgilityPack.

It will get product prices.
But when product stock is 0. They are closing the page.

My program adding prices to listbox. When page giving 404 It should add empty listbox item.

Is there any way to make program simpler? I can't use same Variables at the same button. I'm adding same code changing the numbers (6).

WebRequest SiteyeBaglantiTalebi06 = HttpWebRequest.Create("https://www.themia.com.tr/The-Mia-Dekor-Mermer-22-Cm-Gri,PR-2432.html");
WebResponse GelenCevap06 = SiteyeBaglantiTalebi06.GetResponse();
StreamReader CevapOku06 = new StreamReader(GelenCevap06.GetResponseStream());

string KaynakKodlar06 = CevapOku06.ReadToEnd();
int IcerikBaslangicIndex06 = KaynakKodlar06.IndexOf("<div class=\"productPrice\">") + 122;
int IcerikBitisIndex06 = KaynakKodlar06.Substring(IcerikBaslangicIndex06).IndexOf("</div>");

listBox3.Items.Add((KaynakKodlar06.Substring(IcerikBaslangicIndex06, IcerikBitisIndex06)));

code formating, basic text editing – Sir l33tname Dec 18 '18 at 14:26 — Sir l33tname, Dec 18 '18 at 14:26

Nimrod Dolev · Answer 1 · 2018-12-19T10:04:42.667

If you cast the WebResponse you got to an HttpWeResponse you can access the StatusCode property - https://learn.microsoft.com/en-us/dotnet/api/system.net.httpwebresponse.statuscode?view=netframework-4.7.2#System_Net_HttpWebResponse_StatusCode;

Just one thing to notice - you can't make HttpWebRequest NOT throw an exception when it recieves a status code that does not indicate success (all the more reason not to use this method). This means you have to be ready to catch the exception that will be thrown.

So in the case of your example, it would be -

WebRequest SiteyeBaglantiTalebi06 = HttpWebRequest.Create("https://www.somesite.com/NotARealPath");
try
{
    WebResponse GelenCevap06 = SiteyeBaglantiTalebi06.GetResponse();
    // do things with the result
}
catch (WebException ex)
{
    using (WebResponse response = ex.Response)
    {
        HttpWebResponse asHttp = (HttpWebResponse)response;
        if (asHttp.StatusCode == System.Net.HttpStatusCode.NotFound)
        {
            // your 404 logic here
        }
        else 
        {
            // your "something went wrong but it's not a 404" logic 
        }
    }
}

As for making the code simpler - it's hard to understand exactly what you mean by that without understanding more about your program and what you're trying to do. In general, here are a few ideas -

Stop using HttpWebRequest - here's a nice articale detailing why, and what are your other (and better) options - https://www.diogonunes.com/blog/webclient-vs-httpclient-vs-httpwebrequest/
Put code that is used again and again in a method instead of just copy/pasting it around - https://learn.microsoft.com/en-us/dotnet/csharp/programming-guide/classes-and-structs/methods
Use the Command pattern to bind your button clicks in a more clever way do you can pass parameters (or at least use the 'simple' method in this SO answer) - Add parameter to Button click event

I'm still getting this error : System.Net.WebException: 'The remote server returned an error: (404) Not Found.' — Fraction, Dec 19 '18 at 08:42
You are correct - I forgot that this behaviour (throwing an exception) can't be turned off (see [here](https://stackoverflow.com/questions/692342/net-httpwebrequest-getresponse-raises-exception-when-http-status-code-400-ba) for another related question). I edited to answer to show how to handle this (but I do still recommend not working with the API at all, there are much better options. — Nimrod Dolev, Dec 19 '18 at 09:53

How to check if it is 404 error page and do something using HtmlAgilityPack

1 Answers1