Why does HttpWebRequest throw an exception instead returning HttpStatusCode.NotFound?

Question

I'm trying to verify the existence of a Url using HttpWebRequest. I found a few examples that do basically this:

HttpWebRequest request = (HttpWebRequest)HttpWebRequest.Create(Url);
request.Method = "HEAD";
using (HttpWebResponse response = request.GetResponse() as HttpWebResponse)
{
    return response.StatusCode;
}

However, if the url is indeed broken, it's not returning a response, it's instead throwing an exception.

I modified my code to this:

try
{
    HttpWebRequest request = (HttpWebRequest)HttpWebRequest.Create(Url);
    request.Method = "HEAD";
    using (HttpWebResponse response = request.GetResponse() as HttpWebResponse)
    {
        return response.StatusCode;
    }
}
catch (System.Net.WebException ex)
{
    var response = ex.Response as HttpWebResponse;
    return response == null ? HttpStatusCode.InternalServerError : response.StatusCode;
}

which seems to finally do what I want.

But I would like to know, why is the request throwing an exception instead of returning the response with a NotFound status code?

score 72 · Accepted Answer · answered Apr 10 '12 at 04:26

72

Ya this can be quite annoying when web pages use status codes heavily and not all of them are errors. Which can make processing the body quite a pain. Personally I use this extension method for getting the response.

public static class HttpWebResponseExt
{
    public static HttpWebResponse GetResponseNoException(this HttpWebRequest req)
    {
        try
        {
            return (HttpWebResponse)req.GetResponse();
        }
        catch (WebException we)
        {
            var resp = we.Response as HttpWebResponse;
            if (resp == null)
                throw;
            return resp;
        }
    }
}

answered Apr 10 '12 at 04:26

Will

10,013
9
45
77

7

While this is the least work to save code using HttpWebRequest/Response from this bad design choice on the .Net Framework authors' part, the correct solution is to use HttpClient, which does not throw on 4xx and 5xx status codes. Exceptions are meant for exceptional situations, and throwing just to catch it and proceed as though it's fine like this is ugly and bad for performance, especially given there's a better option that avoids it entirely. https://msdn.microsoft.com/en-us/library/hh138242(v=vs.118).aspx – Chris Moschini Jun 22 '15 at 22:58
4

this does not appear to be true; I'm using HttpClient in a project and when calling a url that does not exist which returns a 404 status code, the client is throwing an exception instead of returning a response with the 404 status code. is there an extra step in using httpclient to prevent this? – SelAromDotNet Apr 12 '17 at 17:17
In my case HttpClient throws exception on 400 Bad Request. – Mariusz Dec 19 '18 at 13:43
In my case an `AggregateException` was thrown and the `InnerException` was a `WebException`. – sdgfsdh Jun 25 '20 at 15:42

score 1 · Answer 2 · answered Apr 10 '12 at 00:57

1

Why not? They're both valid design options, and HttpWebRequest was just designed to work this way.

answered Apr 10 '12 at 00:57

Mahmoud Al-Qudsi

28,357
12
85
125

As long as you can read the response headers and body when the code is 4xx – Ian Boyd Apr 10 '12 at 00:59
6

I guess I was confused, because none of the code samples I saw for this took this into consideration. many didn't even have a try/catch, and I was wondering if perhaps I missed something and there is a way to get the status without throwing an exception. it just seems counterintuitive to throw an entire exception if the statuscode is designed to handle such a status – SelAromDotNet Apr 10 '12 at 01:07
1

Yeah, it always leads to interesting results when you think that "of course xxx tested the line of code he put on his website!" and you turn out to be mistaken :) – Mahmoud Al-Qudsi Apr 10 '12 at 01:35
9

But Microsoft recommends to avoid exception where possible as it can negatively impact performance. http://msdn.microsoft.com/en-us/library/ms229009.aspx – Ramesh Apr 10 '12 at 04:59
13

this is another reason I believe this to be unintuitive. if the response can capture 404 as a status code then I don't think that should be considered an exception. the request completed, it just didn't find anything. – SelAromDotNet Apr 12 '13 at 15:48
1

@MahmoudAl-Qudsi, I'm glad they didn't decide to return exception for 3xx and 20x too, just because you can catch them later! 404 is not an exception and clearly is not an exception on client side, but server side which is another machine which need to handle the exception. and exception on my side means I did something wrong! – AaA Nov 08 '17 at 10:43

Pishang Ujeniya · Answer 3 · 2022-11-23T03:05:32.710

Just like @Will, I wrote similar extension method to get the response content in string from WebException.


        /// <summary>
        /// Reads Response content in string from WebException
        /// </summary>
        /// <param name="webException"></param>
        /// <returns></returns>
        public static (HttpStatusCode statusCode, string? responseString) GetResponseStringNoException(this WebException webException)
        {
            if (webException.Response is HttpWebResponse response)
            {
                Stream responseStream = response.GetResponseStream();

                StreamReader streamReader = new(responseStream, Encoding.Default);

                string responseContent = streamReader.ReadToEnd();
                HttpStatusCode statusCode = response.StatusCode;

                streamReader.Close();
                responseStream.Close();
                response.Close();

                return (statusCode, responseContent);
            }
            else
            {
                return (HttpStatusCode.InternalServerError, null);
            }
        }

The above code is non-optimised solution.

Why does HttpWebRequest throw an exception instead returning HttpStatusCode.NotFound?

3 Answers3

Linked