C# WebBrowser control -- Get Document Elements After AJAX?

Question

I'm writing an application that uses the WebBrowser control to view web content that can change with AJAX that adds new content/elements. I can't seem to get at the new elements any way I've tried. BrowserCtl.DocumentText doesn't have the up-to-date page and of course it's not in "view source" either.

Is there some way to get this new data using this control? :( Please help. Thanks!

IE:

Browser.Navigate("www.somewebpagewithAJAX.com");
//Code that waits for browser to finish...
...
//WebBrowser control has loaded content and AJAX has loaded new content
// (is visible at runtime on form) but can't see them in Browser.Document.All
// or Browser.DocumentText :(

Answer 1

I solved the problem for me.

the key is, attaching a handler for onPropertyChanged event of the div element which is being populated via ajax call.

HtmlElement target = webBrowser.Document.GetElementById("div_populated_by_ajax");

if (target != null)
{
      target.AttachEventHandler("onpropertychange", handler);
}

and finally,

private void handler(Object sender, EventArgs e)
{
      HtmlElement div = webBrowser.Document.GetElementById("div_populated_by_ajax");
      if (div == null) return;
      String contentLoaded = div.InnerHtml; // get the content loaded via ajax
}

Answer 2

using System;
using System.Windows.Forms;

namespace WebBrowserDemo
{
    class Program
    {
        public const string TestUrl = "http://www.w3schools.com/Ajax/tryit_view.asp?filename=tryajax_first";

        [STAThread]
        static void Main(string[] args)
        {
            WebBrowser wb = new WebBrowser();
            wb.DocumentCompleted += new WebBrowserDocumentCompletedEventHandler(wb_DocumentCompleted);
            wb.Navigate(TestUrl);

            while (wb.ReadyState != WebBrowserReadyState.Complete)
            {
                Application.DoEvents();
            }

            Console.WriteLine("\nPress any key to continue...");
            Console.ReadKey(true);
        }

        static void wb_DocumentCompleted(object sender, WebBrowserDocumentCompletedEventArgs e)
        {
            WebBrowser wb = (WebBrowser)sender;

            HtmlElement document = wb.Document.GetElementsByTagName("html")[0];
            HtmlElement button = wb.Document.GetElementsByTagName("button")[0];

            Console.WriteLine(document.OuterHtml + "\n");

            button.InvokeMember("Click");

            Console.WriteLine(document.OuterHtml);           
        }
    }
}

Answer 3

You will need to use DOM for it. Cast WebBrowser.Document.DomDocument to IHTMLDocument?. You will have to import some COM interfaces or Microsoft.mshtml assembly.

Have a look to http://msdn.microsoft.com/en-us/library/aa752641(VS.85).aspx for more details.

Answer 4

I assume that since you're reading content which is generated from Ajax requests that you require the user to progress the application to a point where the relevant data is loaded, at which point you run code to read the data.

If that's not the case, you'll need to automate this process, generating the click events which build out the DOM nodes you're interested in reading. I do this somewhat commonly with the WebBrowser control and tend to write that layer of functionality in Javascript and call it with .InvokeScript(). Another route would be to find the nodes which fire the Ajax functionality from C# and manually trigger their click events:

HtmlElement content = webMain.Document.GetElementById("content");
content.RaiseEvent("onclick");

An important aspect to note in the script above is the fact that you can interact with DOM nodes naively in C# if you accept and work around the limitations of the HtmlElement object type.

Answer 5

How about running javascript to caption the element and displaying it in a new window?

I haven't tested it out but it may work.

(WebBrowser)w.Navigate("javascript:GetElementById('div').innerHtml;", true);

The true attribute to open the return in a new windows. (Or a frame or maybe you can find a better way)

To capture the NewWindow event you'll need to reference the SHDocVw.dll which is in your Windows/System32 folder. Then you can cast your WebBrowser Control like this:

SHDocVw.WebBrowser_V1 browser = (SHDocVw.WebBrowser_V1)(WebBrowser)w.ActiveXInstance;

You can have it close right away after storing the response. Well good luck and let me know how it goes.

Answer 6

Is the information not in the WebBrowser.DocumentText? http://msdn.microsoft.com/en-us/library/system.windows.forms.webbrowser.documenttext.aspx

Answer 7

Do you control the web page?

If so, here's a blog post that explains how: http://www.palladiumconsulting.com/blog/sebastian/2007/04/ultimate-intranet-toy.html

If not, there's probably a solution but I can't help you, sorry.