1

I am using this code:

HttpWebResponse objHttpWebResponse = (HttpWebResponse)objHttpWebRequest.GetResponse();
return new StreamReader(objHttpWebResponse.GetResponseStream()).ReadToEnd();

I get the page content successfully, but my problem is that there are some dynamic content that are populated by javascript functions on the page and it seems that the content is fetched before those functions finished executing, so those parts of the page are returned not populated with data, is there any way to solve this "Wait for page until it's completely loaded including all contents".

Edit:

Regarding "@ElDog" answer, i tried the following code but with no luck to:

        WebBrowser objWebBrowser = new WebBrowser();
        objWebBrowser.DocumentCompleted += objWebBrowser_DocumentCompleted;
        objWebBrowser.Navigate(url);

and at the document complete event i executed the following code:

string content = ((WebBrowser)(sender)).Document.Body.InnerHtml;

But still the javascript functions didn't execute.

ykh
  • 1,775
  • 3
  • 31
  • 57
  • I would suggest you to refer the following Links [Reference Link 1](http://stackoverflow.com/questions/3799242/asynchronous-webrequests-using-c-sharp) [Reference Link 2](http://msdn.microsoft.com/en-IN/library/system.net.webrequest.getresponse(v=vs.71).aspx) [Reference Link 3](http://stackoverflow.com/questions/10565090/getting-the-response-of-a-asynchronous-httpwebrequest) – RajeshKdev May 27 '13 at 09:41
  • RJK, your examples explain how to run http request asynchronously, but they wouldn't solve the pb. Asynchronous or not, HttpWebRequest is not going to run java scripts. – ElDog May 27 '13 at 09:52

1 Answers1

1

HttpWebRequest is not going to execute java scripts at all. It just gives you what a web browser gets in response. To execute java scripts you would need a web browser emulation in your code.

ElDog
  • 1,230
  • 1
  • 10
  • 21
  • I am using WinForms, you mean give my url to a browser control and let it finish loading then get the page content or is there any other way ? – ykh May 27 '13 at 09:40
  • Automating web browser control and getting the parsed result out of it could be a solution though I never did it myself. I did server-side webpages parsing some time ago and I really missed "non-UI" browser which could run java scripts and other dynamic stuff on a page. – ElDog May 27 '13 at 09:43