Can't get ResponseStream from WebException

Question

I have a desktop client, that communicates with serverside via Http. When server has some issues with data processing it returns description of an error in JSON in Http response body with proper Http-code (mainly it is HTTP-400).

When i read HTTP-200 response everithing's fine and this code works:

 using (var response = await httpRequest.GetResponseAsync(token))
                {
                    using (var reader = new StreamReader(response.GetResponseStream(), Encoding.GetEncoding("utf-8")))
                    {
                        return await reader.ReadToEndAsync();
                    }
                }

But when an error occures and WebException is thrown and caught there is this code:

 catch (WebException ex)
            {
                 if (ex.Status == WebExceptionStatus.ProtocolError)
                {                   
                    using (var response = (HttpWebResponse) ex.Response)
                    {
                        using (var stream = response.GetResponseStream())
                        {
                            using (var reader = new StreamReader(stream, Encoding.GetEncoding("utf-8")))
                            {
                                var json = reader.ReadToEnd();
                            }
                        }
                    }
                }
            }

I have already done something to it to maybe make it work, but the next happens: response.ContentLength is valid (184) but stream.Length is 0 and after that i can't read json (it's "") I don't even know where to look, because everything looks like it should work.

What might be the problem?

Answer 1

After a month of almost everyday thinking I've found workaround.

The thing was that WebException.Response.GetResponseStream() returns not exactly the same stream that was obtained during request (can't find link to msdn right now) and by the time we get to catch the exception and read this stream the actual response stream is lost (or something like that, don't really know and was unable to find any info on the net, except looking into CLRCore which is now opensource).

To save the actual response until catching WebException you must set KeepAlive Property on your HttpRequest and voila, you get your response while catching exception. So the working code looks like that:

            try
            {
                var httpRequest = WebRequest.CreateHttp(Protocol + ServerUrl + ":" + ServerPort + ServerAppName + url);

                if (HttpWebRequest.DefaultMaximumErrorResponseLength < int.MaxValue)
                    HttpWebRequest.DefaultMaximumErrorResponseLength = int.MaxValue;

                httpRequest.ContentType = "application/json";
                httpRequest.Method = method;
                var encoding = Encoding.GetEncoding("utf-8");

                if (httpRequest.ServicePoint != null)
                {
                    httpRequest.ServicePoint.ConnectionLeaseTimeout = 5000;
                    httpRequest.ServicePoint.MaxIdleTime = 5000;
                }
                //----HERE--
                httpRequest.KeepAlive = true;
                //----------
                using (var response = await httpRequest.GetResponseAsync(token))
                {
                    using (var reader = new StreamReader(response.GetResponseStream(), encoding))
                    {                       
                        return await reader.ReadToEndAsync();
                    }
                }
            }
            catch (WebException ex)
            {
                if (ex.Status == WebExceptionStatus.ProtocolError)
                {
                    using (var response = (HttpWebResponse)ex.Response)
                    {
                        using (var stream = response.GetResponseStream())
                        {
                            using (var reader = new StreamReader(stream, Encoding.GetEncoding("utf-8")))
                            {
                                return reader.ReadToEnd();
                                //or handle it like you want
                            }
                        }
                    }
                }
            }

I don't know if it is good to keep all connection alive like that, but since it helped me to read actual responses from server, i think it might help someone, who faced the same problem.

EDIT: Also it is important not to mess with HttpWebRequest.DefaultMaximumErrorResponseLength.

Answer 2

I remember facing similar issue before and there was something related to setting the stream's position. Here is one of my solutions for reading webResponse that worked for me earlier. Please try if similar approach works for you:-

private ResourceResponse readWebResponse(HttpWebRequest webreq)
    {
        HttpWebRequest.DefaultMaximumErrorResponseLength = 1048576;
        HttpWebResponse webresp = null;// = webreq.GetResponse() as HttpWebResponse;
        var memStream = new MemoryStream();
        Stream webStream;
            try
            {
                webresp = (HttpWebResponse)webreq.GetResponse();
                webStream = webresp.GetResponseStream();
                byte[] readBuffer = new byte[4096];
                int bytesRead;
                while ((bytesRead = webStream.Read(readBuffer, 0, readBuffer.Length)) > 0)
                    memStream.Write(readBuffer, 0, bytesRead);
            }
            catch (WebException e)
            {
                var r = e.Response as HttpWebResponse;
                webStream = r.GetResponseStream();
                memStream = Read(webStream);
                var wrongLength = memStream.Length;
            }


            memStream.Position = 0;
            StreamReader sr = new StreamReader(memStream);
            string webStreamContent = sr.ReadToEnd();

            byte[] responseBuffer = Encoding.UTF8.GetBytes(webStreamContent);
//......
//.......

Hope this helps!