87

I see many examples but all of them read them into byte arrays or 256 chars at a time, slowly. Why?

Is it not advisable to just convert the resulting Stream value into a string where I can parse it?

Joan Venge
  • 315,713
  • 212
  • 479
  • 689

5 Answers5

151

You can use StreamReader.ReadToEnd(),

using (Stream stream = response.GetResponseStream())
{
   StreamReader reader = new StreamReader(stream, Encoding.UTF8);
   String responseString = reader.ReadToEnd();
}
KV Prajapati
  • 93,659
  • 19
  • 148
  • 186
61

You should create a StreamReader around the stream, then call ReadToEnd.

You should consider calling WebClient.DownloadString instead.

SLaks
  • 868,454
  • 176
  • 1,908
  • 1,964
  • 1
    Thanks SLaks, checking DownloadString now. Actually I am gettng different source code strings compared to my old app where it uses browser.DocumentStream using Winforms Browser control. You think DownloadString would fix it? I can create a new question if it's not as straightforward. – Joan Venge Sep 25 '11 at 02:54
  • 5
    You either have a encoding issue or you need to set a `User-Agent`. – SLaks Sep 25 '11 at 02:59
  • Thanks SLaks, I use DownloadString now, and it's better you are right. Now the result differs slightly, shouldn't make a difference but I get stuff like ` – Joan Venge Sep 25 '11 at 03:01
  • 2
    That sounds weird; it might be normalized by IE. – SLaks Sep 25 '11 at 03:02
  • Thanks SLaks, I noticed IE, chrome, FF all gives slightly different results. So DownloadString still uses IE behind the scenes? Or is it more back more direct no-IE dependent method? – Joan Venge Sep 25 '11 at 03:03
  • 3
    `WebClient` and `HttpWebRequest` use raw HTTP with no browser involved. If different browsers show different **View Source** s, it's a `User-Agent` issue. – SLaks Sep 25 '11 at 03:09
  • The answer is valid but comment about WebClient is not. You get WebResponse also from WebException.Response where you get the complete response from remote server.. – Evžen Černý Aug 18 '23 at 11:52
14

As @Heinzi mentioned the character set of the response should be used.

var encoding = response.CharacterSet == ""
    ? Encoding.UTF8
    : Encoding.GetEncoding(response.CharacterSet);

using (var stream = response.GetResponseStream())
{
    var reader = new StreamReader(stream, encoding);
    var responseString = reader.ReadToEnd();
}
Uwe Keim
  • 39,551
  • 56
  • 175
  • 291
Richard Schneider
  • 34,944
  • 9
  • 57
  • 73
  • This should be the accepted answer. One bug though, change response.CharacterSet == "" to string.IsNullOrEmpty(response.CharacterSet) – nivs1978 Apr 20 '21 at 06:51
5

Richard Schneider is right. use code below to fetch data from site which is not utf8 charset will get wrong string.

using (Stream stream = response.GetResponseStream())
{
   StreamReader reader = new StreamReader(stream, Encoding.UTF8);
   String responseString = reader.ReadToEnd();
}

" i can't vote.so wrote this.

Howard Lou
  • 59
  • 1
  • 1
2

You can create a StreamReader around the stream, then call StreamReader.ReadToEnd().

StreamReader responseReader = new StreamReader(request.GetResponse().GetResponseStream());
var responseData = responseReader.ReadToEnd();
timschoen
  • 171
  • 2
  • 9
Mari
  • 237
  • 1
  • 4
  • 17