How to get rendered html (processed by Javascript) in WebBrowser control?

Question

I have an ASP.NET page and some custom class that fetches a specified webpage and returns that page body back.

protected String GetHtml()
{
    Thread thread = new Thread(new ThreadStart(GetHtmlWorker));
    thread.SetApartmentState(ApartmentState.STA);
    thread.Start();
    thread.Join();
    return docHtml;
}

protected void GetHtmlWorker()
{
    using (WebBrowser browser = new WebBrowser())
    {
        browser.ScriptErrorsSuppressed = true;
        browser.Navigate(_url);
        // Wait for control to load page
        while (browser.ReadyState != WebBrowserReadyState.Complete)
            Application.DoEvents();
        docHtml = browser.DocumentText;
    }
}

But what I need is to get DOM HTML instead of the page source because I do some extra operations over DOM by jQuery.

So what you need is the DOM html after javascript had modified it? Do you want it as a string? — Răzvan Flavius Panda, Sep 07 '11 at 14:24

score 17 · Accepted Answer · answered Sep 09 '11 at 08:51

Here is one solution I found to get to the rendered HTML(DOM) after javascript was run:

Place a WebBrowser control named webBrowser1 on the Form of class Form1.

[Form1.cs[Design]]

Then for code use:

[Form1.cs]

using System;
using System.Runtime.InteropServices;
using System.Windows.Forms;

namespace WebBrowserTest
{
    public partial class Form1 : Form
    {
        public Form1()
        {
            InitializeComponent();
            this.webBrowser1.ObjectForScripting = new MyScript();
        }

        private void Form1_Load(object sender, EventArgs e)
        {
            webBrowser1.Navigate("http://localhost:6489/Default.aspx");
        }

        private void webBrowser1_DocumentCompleted(object sender, WebBrowserDocumentCompletedEventArgs e)
        {
            webBrowser1.Navigate("javascript: window.external.CallServerSideCode();");
        }

        [ComVisible(true)]
        public class MyScript
        {
            public void CallServerSideCode()
            {
                var doc = ((Form1)Application.OpenForms[0]).webBrowser1.Document;
            }
        }
    }
}

Change the webBrowser1.Navigate("http://localhost:6489/Default.aspx") parameter in Form1_Load to the page whose DOM after being processed by javascript you wish to obtain.

You can access the modified DOM in the CallServerSideCode() method, for example:

doc.GetElementById("myDataTable");

Or you can access the rendered HTML like this:

var renderedHtml = doc.GetElementsByTagName("HTML")[0].OuterHtml;

that's a great answer Răzvan Panda!!! I used your code to read some cold fusion dynamically render page and it worked until I had to rebuild my syetem. NOW, I cannot see teh actual rendered data (inner/outer)html fail to return what I see. is there another place or method that I can use to retireve it?? — CocoaNewBee, Aug 09 '12 at 15:57
@CocoaNewBee: I don't know of any other way, I learned this way of accessing it by research. The way you described it, is like, you couldn't get it in another way either. — Răzvan Flavius Panda, Sep 21 '12 at 08:23
Why not do it in webBrowser1_DocumentCompleted like webBrowser1.Document.GetElementsByTagName("HTML")[0].OuterHtml — TechGeorgii, Jun 05 '13 at 05:36
Worked for me thanks. I wish Microsoft had made their documentation clear instead of suggesting that the DocumentText property would return the dom. — Alan Gee, Aug 26 '16 at 14:22

jimmyjudas · Answer 2 · 2014-12-04T15:56:13.283

6

As George said in one of the comments, in theory you can just get the DOM in webBrowser1_DocumentCompleted by just using:

webBrowser1.Document.GetElementsByTagName("HTML")[0].OuterHtml;

edited Dec 04 '14 at 15:56

answered Dec 04 '14 at 15:26

jimmyjudas

581
7
13

Thanks, it include all html of ajax – Dũng IT Aug 17 '18 at 09:05

score 2 · Answer 3 · edited Feb 07 '15 at 02:02

First a little background. I have been trying to scrape information from a web page. The content of this webpage is dynamic. What I mean by dynamic is that the web page loads more information as you scroll down to the bottom of the page. The HTML content changes as you scroll to the bottom of the page. Unfortunately the Web Browser Object does not update this information automatically. It still has the original document that it first loaded via the webbrowser.navigate function. The updated information is available to the HTMLElementCollection.

The following code did not work for me.

webBrowser1.Document.GetElementsByTagName("HTML")[0].OuterHtml

I broke up the above statement as follows

    Dim eCollections As HtmlElementCollection
    Dim strDoc As String
    eCollections = WB.Document.GetElementsByTagName("HTML")
    strDoc = eCollections(0).OuterHtml

Worked like a charm. Hope this helps someone too.

score 0 · Answer 4 · answered Sep 30 '15 at 22:22

0

Another way would be to set a timer on the form, then when the timer hits, the page will have re-rendered and you can parse the page.

answered Sep 30 '15 at 22:22

JeffT

181
2
9

score 0 · Answer 5 · answered Jun 29 '17 at 15:25

0

You can get

webBrowser1.Document.Body.OuterHtml

answered Jun 29 '17 at 15:25

ngochoaitn

27
4

How to get rendered html (processed by Javascript) in WebBrowser control?

5 Answers5

Linked