HttpResponseMessage Content string appears to be wrong encoding

Question

I'm using a console application to communicate with a third party website. In order to determine if a form submission completed successfully I would like to query the response message to determine if the submit button is still on the page.

The response headers in Fiddler look like this:

HTTP/1.1 200 OK
Cache-Control: private
Content-Type: text/html; charset=utf-8
Vary: Accept-Encoding
Server: Microsoft-IIS/7.5
X-AspNet-Version: 4.0.30319
X-Powered-By: ASP.NET
Date: Fri, 08 May 2015 19:31:34 GMT
Content-Length: 93993

My code looks like this:

var enc = Encoding.UTF8;
using (Stream responseStream =  responseMessage.Content.ReadAsStreamAsync().Result)
            {
                using (var rd = new StreamReader(responseStream, enc))
                {
                    var stringContent = rd.ReadToEnd();
                    if (!stringContent.Contains("submitButtonTag"))
                        result = true;

                }
            }

However when I view the returned string in Visual Studio the characters look like this:

�\b\0\0\0\0\0\0�\a`I�%&/m�{J�J��t�\b�`$ؐ@������iG#)�*��eVe]f@�흼��{���

I would just run through all the way to the submit page to see what happens but I have a limited number of test forms I can submit. Is this just the way that Visual Studio renders these? Is there a way I can get this response to be readable characters that I can compare against?

Just a guess - even if you don't request it, and even if they don't tell you in the response, some sites might return GZipped content. Try taking the raw bytes of the response and see if it's actually a valid gzip document, and if so, then you can add code to unzip it before attempting to read the string. — Joe Enos, May 08 '15 at 21:38
I think this might be of help: [.NET: Is it possible to get HttpWebRequest to automatically decompress gzip'd responses?](http://stackoverflow.com/a/2815876/1115360) — Andrew Morton, May 08 '15 at 21:52

Aaron · Answer 1 · 2015-05-13T12:39:21.297

After the comment from @JoeEnos I was able to simply add a GZip decompression before reading the stream. Code is below.

var enc = Encoding.UTF8;

using (Stream responseStream =  responseMessage.Content.ReadAsStreamAsync().Result)
{
    using (var decompressedStream = new GZipStream(responseStream, CompressionMode.Decompress))
    {
        using (var rd = new StreamReader(decompressedStream, enc))
        {
            var stringContent = rd.ReadToEnd();
            if (!stringContent.Contains("submitButtonTag"))
                result = true;

        }
    }                   
}

HttpResponseMessage Content string appears to be wrong encoding

1 Answers1