Does GetResponseStream()).ReadToEnd() read only 182955 characters

Question

I am doing an HTTP POST and getting a huge XML back in the response . I am seeing that the xml gets truncated at 182956 th charcater and hence I am not able to Deserialize the response . Is there a way I can read the entire content ? Thanks in advance for your help .

string myresponse = string.Empty();
 HttpWebResponse httpmyResponse = (HttpWebResponse)myrequest.GetResponse();
 response = new StreamReader(httpmyResponse.GetResponseStream()).ReadToEnd();

Content-Length: 444313

Content-Type: application/xml

Have you seen [this SO question](http://stackoverflow.com/questions/4918107/system-net-httpwebresponse-getresponsestream-returns-truncated-body-in-webexce) ? — Michael, Sep 15 '14 at 21:07
@michaelmoore The question talks about the error response. I am seeing when Status code is 200, valid response — user1110790, Sep 16 '14 at 17:20
Content-Length is in bytes, you are counting characters. The most obvious source of the problem is on the other end of the wire. — Hans Passant, Sep 16 '14 at 23:26

score 0 · Answer 1 · edited May 23 '17 at 11:49

SO post I referred to in comments might actually solve your problem. In particular if you set DefaultMaximumErrorResponseLength to bigger value it might help. Internally, here how ResponseStream is being created

   private Stream MakeMemoryStream(Stream stream) {
           // some code emitted here     
            SyncMemoryStream memoryStream = new SyncMemoryStream(0);      // buffered Stream to save off data
            try {
                //
                // Now drain the Stream
                //
                if (stream.CanRead) {
                    byte [] buffer = new byte[1024];
                    int bytesTransferred = 0;

                    int maxBytesToBuffer = (HttpWebRequest.DefaultMaximumErrorResponseLength == -1)?buffer.Length:HttpWebRequest.DefaultMaximumErrorResponseLength*1024;
                    while ((bytesTransferred = stream.Read(buffer, 0, Math.Min(buffer.Length, maxBytesToBuffer))) > 0)
                    {
                        memoryStream.Write(buffer, 0, bytesTransferred);
                        if(HttpWebRequest.DefaultMaximumErrorResponseLength != -1)
                            maxBytesToBuffer -= bytesTransferred;
                    }
                }
                memoryStream.Position = 0;
            }
            catch {
            }

            // some other code

            return memoryStream;
        }

Important members here are stream that is response stream, and memoryStream - that is response stream that you're going to get back as a result to a method call GetResposneStream(). As you can see, before reading stream, method sets maxBytesToBuffer equal to DefaultMaximumErrorResponseLength*1024, if DefaultMaximumErrorResponseLength is not equal to -1, otherwise to the length of buffer which is 1024. Then, in the while loop, it reads stream, and on each iteration decreases maxBytesToBuffer by amount of bytes read (maxBytesToBuffer -= bytesTransferred). Now lets consider both cases

DefaultMaximumErrorResponseLength is -1, stream length is 444313. In this case maxBytesToBuffer will be equal to buffer.Length, which is 1024. So it will read 1024 bytes, as a result bytesTransferred will be 1024, after first iteration, maxBytesToBuffer will become 0 (because of maxBytesToBuffer -= bytesTransferred), so next time it will read 0 bytes, and exit while loop, so you will have only 1024 bytes read from your entire stream.
DefaultMaximumErrorResponseLength is 1024, stream length is 444313. In this case maxBytesToBuffer will be equal to DefaultMaximumErrorResponseLength*1024 = 1048576. Again entering while loop first time, it will read 1024 (because of Math.Min(buffer.Length, maxBytesToBuffer)). On each iteration it will decrease maxBytesToBuffer by 1024, so while loop can iterate at least 1024 times, each time reading 1024 bytes. After roughly 433 iterations (that is your content length 444313/1024 = 433.8) it should read all of your content in the stream.

Having said this, I would first check what's the value of DefaultMaximumErrorResponseLength and do the math (as I've done previously), and see if that is root cause of your problem or not.

Code was taken from MS Reference Source web site

Does GetResponseStream()).ReadToEnd() read only 182955 characters

1 Answers1