0

I am working on a little project and it involves getting data from a website to calculate stuff.

I tried

    HttpWebRequest request = (HttpWebRequest)WebRequest.Create(url);
    HttpWebResponse response = (HttpWebResponse)request.GetResponse();
    StreamReader sr = new StreamReader(response.GetResponseStream());
    rtb_data.Text = sr.ReadToEnd();
    sr.Close();

But this way, I dont get the full source code, like I do when I open the developer kit on chrome for instance (F12).

Example:

    <div id="live-match" class="panel-container" data-match-status-code="6" data-mode="3">

I get:

    <div id="live-match" class="panel-container">

in that div are many important information but I dont get the classes at all.

I am no pro with C# or dynamic webpages but since I am able to see the real source code with the developer kit, it should be possible to get it in C# too, right?

My last solution would be to work with the copy/paste source code and work from there but that would be really slow since I wanna automatically calculate stuff.

Do you guys have any idea how I can get the full source code? Do I have to temp download the content maybe? If yes, how?

I3LaCkPeArL
  • 41
  • 1
  • 4
  • Presumably, the developer tools are showing you content that has been generated from javascript -- thus, without running you're own interpreter you're not going to get it. If it's just for personal use, a chrome plugin might be the way to go? – Lincoln Green Oct 06 '16 at 19:38
  • There are many ways to render page (execute javascript). One of the ways is described here: http://stackoverflow.com/q/24288726/5311735 – Evk Oct 06 '16 at 19:39
  • That link looks promising, thank you for that link. :) – I3LaCkPeArL Oct 06 '16 at 19:51
  • Be sure to look at "easier method" at the bottom of accepted answer there. – Evk Oct 06 '16 at 19:52

1 Answers1

-1

The problem you have is that you don't want the source of the web page (that is what you get when you do your simple web request) You seem to actually be after the DOM that is generated at runtime.

Unfortunately, unless you are willing to write your own browser...

Scott Perham
  • 2,410
  • 1
  • 10
  • 20
  • So, the webbrowser control will mostly allow you to at least get the active webpage (including the dom manipulations) in winforms, provided it's compatibel with the IE renderer. Also the link given by Evk seems quite handy for exactly such purposes, it's not really that hard to do this... – Icepickle Oct 06 '16 at 20:05