How to Handle Threading for WebBrowser Control

Question

I am using BrowserControl to navigate to range of web pages on a site and then parse the html and extract information about books etc… I am having problems related (I think) to threading…

I have something like this.

// MAIN LOOP
for (int i = 0; i < NumberOfPages; i++)
 {
  WebBrowser.Navigate("http://AWebSite/" + NumberOfPages.ToString());
 }

// HANDLE ON_LOADED EVENT
 void WebBrowser_LoadCompleted(object sender, NavigationEventArgs e)
    {
   // Retrieve HTMLDocument, Parse it etc
    }

Now since it takes a few seconds for the event to fire after the control navigates to a page, I have one of two options:

OPTION1 Wait a few seconds in my main loop, like this:

for (int i = 0; i < NumberOfPages; i++)
{
  WebBrowser.Navigate("http://www.mysite.com"); 

// wait for 5 seconds
DateTime wait = new DateTime();
while (new DateTime().Ticks < wait.Ticks + 5000)  
    {
     // not sure if I need do events here         
    }
}

OPTION2 Another idea is to a Global Variable as a (Boolean) Flag to indicate to the event handler that the page is still downloading (the flag is set to busy in the main look and then reset and then reset after after handling the html returned).

I have a feeling both of these approaches are clumsy and really that there is a better way is to somehow handle these two things (running on different threads?)

score 1 · Answer 1 · answered Nov 05 '10 at 19:14

1

Yeah, a delay is clumsy - it may take longer than that or whatever.

Do you need the WebBrowser control? It looks like you're doing some batch processing. If that's so, the System.Net.WebClient may work for you. It has blocking and asynchronous methods - .DownloadData and .DoanloadDataAsync.

I can probably dig up some code if you need it, but a quick search shows some examples out there.

answered Nov 05 '10 at 19:14

Mike Hildner

1,154
1
11
29

Hi Mike, Any chance to post a simple example of how to iterate through multiple navigating to multiple urls and managing to handle the returned html? – underwater Nov 06 '10 at 17:25
Let me give you a link with a code sample - not at my dev box right now - http://msdn.microsoft.com/en-us/library/xz398a3f.aspx . You can put that in a loop like your original post if you want. If you have issues, let us know. – Mike Hildner Nov 07 '10 at 19:34

score 0 · Answer 2 · edited May 23 '17 at 12:11

You can do this by misuing iterators, as I described here.

For example:

interface IAction { void Execute(Action callback); }

public static void ExecAction(IEnumerator<IAction> enumerator) {
    if (enumerator.MoveNext())
        enumerator.Current.Execute(() => ExecAction(enumerator));
}

class WaitForLoad : IAction {
    void IAction.Execute(Action callback) {
       //Handle the LoadCompleted event and call callback
    }
}

IEnumerator<IAction> YourMethod() { 
    ...
    for (int i = 0; i < NumberOfPages; i++) {
        WebBrowser.Navigate("http://AWebSite/" + NumberOfPages.ToString());
        yield return new WaitForLoad();
    }
    ...
}

You can make this more sophisticated by having WaitForLoad check that the correct page has loaded.

Thanks, tried to read through your example but it looks way above my head :-( Is there a simpler way to do this, given that I don't mind if only one a single url is fetched at a time ? — underwater, Nov 06 '10 at 17:29

How to Handle Threading for WebBrowser Control

2 Answers2

Linked