Background:
I have an html page that has a few DIVs and tables that are already in the document DOM. when the url of this page is invoked, a JavaScript code fills and populates the contents of this page. This may taake 2~3 seconds, but eventually gets done nicely.
Problem: I would like to invoke the URL of this page in C#, to get its contents in a nice html string.
I have tried many tricks that I learned from SO. However, all of them only return the bare-bone structure of the page without any content. These include:
Now my question is that is there a nice way of waiting for all the contents to load when invoking a URL?
some options to me seem like to be along:
Is there a way to tell one of these class to wait a certain amount of time before downloading the contents?, or:
Is there another Event on WebBrowser class that I can hook so that it waits for all the content to load?
Thanks in advance.
Edit 3:
Tried this other solution using HttpWebRequests, but got this error: 'HttpWebRequest.HttpWebRequest()' is obsolete: 'This API supports the .NET Framework infrastructure and is not intended to be used directly from your code.'
Edit 2: Even this other WebBrowser solution does not work!!
It raises the JS error "object does not support "CallServerSideCode"
Edit 1: as Austin asked, I am including some of the code that I have tried:
using System;
using System.Collections.Generic;
using System.ComponentModel;
using System.Data;
using System.Drawing;
using System.Linq;
using System.Text;
using System.Threading.Tasks;
using System.Windows.Forms;
using System.Diagnostics;
using System.Net;
using System.Net.Http;
using System.Threading;
using System.IO;
namespace WindowsFormsApp1
{
public partial class Form1 : Form
{
WebBrowser wb = new WebBrowser();
public Form1()
{
InitializeComponent();
wb.DocumentCompleted += new WebBrowserDocumentCompletedEventHandler(wb_DocumentCompleted);
}
private void button1_Click(object sender, EventArgs e)
{
using(var webc = new WebClient())
{
var json = webc.DownloadString(textBox1.Text);
Debug.WriteLine("WebClient: \r\n" + json.ToString() + " \r\n\r\n\r\n");
}
}
private void button2_Click(object sender, EventArgs e)
{
WebRequest request = WebRequest.Create(textBox1.Text);
// If required by the server, set the credentials.
request.Credentials = CredentialCache.DefaultCredentials;
// Get the response.
Thread.Sleep(5000);
WebResponse response = request.GetResponse();
// Display the status.
Console.WriteLine(((HttpWebResponse)response).StatusDescription);
// Get the stream containing content returned by the server.
Stream dataStream = response.GetResponseStream();
// Open the stream using a StreamReader for easy access.
StreamReader reader = new StreamReader(dataStream);
// Read the content.
string responseFromServer = reader.ReadToEnd();
// Display the content.
Console.WriteLine("Web Request: \r\n" + responseFromServer + " \r\n\r\n\r\n");
// Clean up the streams and the response.
reader.Close();
response.Close();
}
private void button3_Click(object sender, EventArgs e)
{
wb.Navigate(new Uri(textBox1.Text));
}
private void wb_DocumentCompleted(object sender, WebBrowserDocumentCompletedEventArgs e)
{
string page = wb.DocumentText;
Console.WriteLine("Web Browser: \r\n" + page + " \r\n\r\n\r\n");
}
}