0

Background:

I have an html page that has a few DIVs and tables that are already in the document DOM. when the url of this page is invoked, a JavaScript code fills and populates the contents of this page. This may taake 2~3 seconds, but eventually gets done nicely.

Problem: I would like to invoke the URL of this page in C#, to get its contents in a nice html string.

I have tried many tricks that I learned from SO. However, all of them only return the bare-bone structure of the page without any content. These include:

Using WebBrowser

Using WebClient-1

Using WebClient-2

Using WebRequest

Now my question is that is there a nice way of waiting for all the contents to load when invoking a URL?

some options to me seem like to be along:

Is there a way to tell one of these class to wait a certain amount of time before downloading the contents?, or:

Is there another Event on WebBrowser class that I can hook so that it waits for all the content to load?

Thanks in advance.


Edit 3:

Tried this other solution using HttpWebRequests, but got this error: 'HttpWebRequest.HttpWebRequest()' is obsolete: 'This API supports the .NET Framework infrastructure and is not intended to be used directly from your code.'


Edit 2: Even this other WebBrowser solution does not work!!

It raises the JS error "object does not support "CallServerSideCode"


Edit 1: as Austin asked, I am including some of the code that I have tried:

using System;
using System.Collections.Generic;
using System.ComponentModel;
using System.Data;
using System.Drawing;
using System.Linq;
using System.Text;
using System.Threading.Tasks;
using System.Windows.Forms;
using System.Diagnostics;
using System.Net;
using System.Net.Http;
using System.Threading;
using System.IO;

namespace WindowsFormsApp1
{
    public partial class Form1 : Form
    {
        WebBrowser wb = new WebBrowser();

        public Form1()
        {
            InitializeComponent();
            wb.DocumentCompleted += new WebBrowserDocumentCompletedEventHandler(wb_DocumentCompleted);
        }

        private void button1_Click(object sender, EventArgs e)
        {
            using(var webc = new WebClient())
            {
                var json = webc.DownloadString(textBox1.Text);
                Debug.WriteLine("WebClient: \r\n" + json.ToString() + " \r\n\r\n\r\n");
            }
        }

        private void button2_Click(object sender, EventArgs e)
        {
            WebRequest request = WebRequest.Create(textBox1.Text);
            // If required by the server, set the credentials.  
            request.Credentials = CredentialCache.DefaultCredentials;
            // Get the response.  
            Thread.Sleep(5000);
            WebResponse response = request.GetResponse();

            // Display the status.  
            Console.WriteLine(((HttpWebResponse)response).StatusDescription);
            // Get the stream containing content returned by the server.  
            Stream dataStream = response.GetResponseStream();
            // Open the stream using a StreamReader for easy access.  
            StreamReader reader = new StreamReader(dataStream);
            // Read the content.  
            string responseFromServer = reader.ReadToEnd();
            // Display the content.  
            Console.WriteLine("Web Request: \r\n" + responseFromServer + " \r\n\r\n\r\n");
            // Clean up the streams and the response.  
            reader.Close();
            response.Close();
        }

        private void button3_Click(object sender, EventArgs e)
        {            
            wb.Navigate(new Uri(textBox1.Text));
        }

        private void wb_DocumentCompleted(object sender, WebBrowserDocumentCompletedEventArgs e)
        {
            string page = wb.DocumentText;
            Console.WriteLine("Web Browser: \r\n" + page + " \r\n\r\n\r\n");
        }


    }
AleX_
  • 508
  • 1
  • 6
  • 20
  • `document.ready()` is called when the page is loaded fully. I think I misundertand your question. – Steve0 Aug 02 '17 at 21:48
  • Document.Ready is not a WebBrowser Event, do you mean in the JS code? – AleX_ Aug 02 '17 at 22:02
  • Put some of the code you have tried in the question... – Austin T French Aug 02 '17 at 22:32
  • Which of your 3 buttons is the closest to working the way you want it to. I am still not clear on what your issue is. – Steve0 Aug 03 '17 at 15:11
  • @Steve, button three with `wb.Document.outerHTML` property. Right now I am blocked by a logic in the JS code that does not work when there are authentication problem – AleX_ Aug 03 '17 at 16:44

0 Answers0