webbrowser auto navigation event

Question

I am using webbrowser to navigate to a website then automating the login. Everything works perfectly up until the point of the comment "Navigating Event" After entering one credential it will login and navigate to another website. After the event none of the code will work as it is not picking up the new site. I am using a waitforpageload() function to let me know when it is completed loading however when I check the url it is still pointing to the original site. Any ideas why it would be doing this and how to possibly get around it?

    Private Property pageready As Boolean = False

    webBrowser1.Navigate("https://www.lamedicaid.com/sprovweb1/provider_login/provider_login.asp")
            waitforpageload()

    Dim allelements As HtmlElementCollection = webBrowser1.Document.All
            For Each webpageelement As HtmlElement In allelements
                'NPI #
                If webpageelement.GetAttribute("name") = "Provider_Id" Then
                    webpageelement.SetAttribute("value", "xxxxxx")
                End If
                'Clicking enter to input NPI
                If webpageelement.GetAttribute("name") = "submit1" Then
                    webpageelement.InvokeMember("focus")
                    webpageelement.InvokeMember("click")
                    waitforpageload()
                End If

                'Navigation event happens here

                'Entering username
                If webpageelement.GetAttribute("name") = "Login_Id" Then
                    webpageelement.SetAttribute("value", "xxxxxxx")
                End If
                'Entering Password
                If webpageelement.GetAttribute("name") = "Password" Then
                    webpageelement.SetAttribute("value", "xxxxxxxxx")
                End If
                'logging in
                If webpageelement.GetAttribute("name") = "submit_button" Then
                    webpageelement.InvokeMember("focus")
                    webpageelement.InvokeMember("click")
                    waitforpageload()
                End If




    #Region "Page Loading Functions"
    Private Sub waitforpageload()
        AddHandler webBrowser1.DocumentCompleted, New WebBrowserDocumentCompletedEventHandler(AddressOf PageWaiter)
        While Not pageready
            Application.DoEvents()
        End While
        pageready = False
    End Sub

    Private Sub PageWaiter(ByVal sender As Object, ByVal e As WebBrowserDocumentCompletedEventArgs)
        If webBrowser1.ReadyState = WebBrowserReadyState.Complete Then
            pageready = True
            RemoveHandler webBrowser1.DocumentCompleted, New WebBrowserDocumentCompletedEventHandler(AddressOf PageWaiter)
        End If
    End Sub
#End Region

First thing, you have way too many `waitforpageload()` method calls. You don't need to wait for anything when you set an element attribute. Then, a consideration: do you need to do this for these two sites only? If so, you just need one event handler for `DocumentComplete()`. Set there a logic to discriminate between sites, maybe using their page title (or some other element), and take action accordingly. — Jimi, Nov 29 '17 at 20:20
@Jimi the second page has a new URL, but when I navigate to it after processing the first one it kicks me out. Treating the navigation as a new page. Also, I have updated the waitforpagetoload() method calls. I will look into utilizing the page title or some other element. — Cole Perrault, Nov 29 '17 at 20:28
@Jimi (and Cole) : Third thing, _**do not use `waitforpageload()`**_! — Visual Vincent, Nov 29 '17 at 20:39
@Cole Perrault "kicks me out" is undefined. Follow this a moment: (1) Navigate 1st URL (...navigating...) => `DocumentComplete()` fired => (2) Set login values->InvokeMember() => LogIn accepted => (...navigating...)=> `DocumentComplete()` fired (3) LandingPage-> New LogIn form-> SetValues-> InvokeMember => LogIn accepted => (...navigating...)=> `DocumentComplete()` (4) LandingPage -> Do whatever you need to do here — Jimi, Nov 29 '17 at 20:48

Visual Vincent · Accepted Answer · 2017-11-29T21:15:38.897

Please, for the love of god, get rid of that waitforpageload() function! Using Application.DoEvents() is BAD PRACTICE and in a loop like that, will utilize 100% of your CPU!

The person who originally wrote that function (it's from another Stack Overflow post) clearly didn't know what he/she was doing at the time. Using Application.DoEvents() creates more problems than it solves, and should NEVER be used in anyone's code (it exists mostly because it is used by internal methods).

Refer to: Keeping your UI Responsive and the Dangers of Application.DoEvents for more info.

The WebBrowser has a dedicated DocumentCompleted event that is raised every time a page (or part of a page, such as an iframe) has been completely loaded.

To make sure that the page really is fully loaded, subscribe to the DocumentCompleted event and check if the ReadyState property is equal to WebBrowserReadyState.Complete.

To be able to run code more "dynamically" when the DocumentCompleted event is raised you can utilize lambda expressions as a way of creating inline methods.

In your case they can be used like this:

'Second step (these must always be in descending order since the first step must be able to reference the second, and so on).
Dim credentialHandler As WebBrowserDocumentCompletedEventHandler = _
    Sub(wsender As Object, we As WebBrowserDocumentCompletedEventArgs)
        'If the WebBrowser HASN'T finished loading, do not continue.
        If webBrowser1.ReadyState <> WebBrowserReadyState.Complete Then Return

        'Remove the event handler to avoid this code being called twice.
        RemoveHandler webBrowser1.DocumentCompleted, credentialHandler

        'Entering username
        If webpageelement.GetAttribute("name") = "Login_Id" Then
            webpageelement.SetAttribute("value", "xxxxxxx")
        End If

        'Entering Password
        If webpageelement.GetAttribute("name") = "Password" Then
            webpageelement.SetAttribute("value", "xxxxxxxxx")
        End If

        'logging in
        If webpageelement.GetAttribute("name") = "submit_button" Then
            webpageelement.InvokeMember("focus")
            webpageelement.InvokeMember("click")
        End If
    End Sub


'First step.
Dim loginHandler As WebBrowserDocumentCompletedEventHandler = _
    Sub(wsender As Object, we As WebBrowserDocumentCompletedEventArgs)
        'If the WebBrowser hasn't finished loading, do not continue.
        If webBrowser1.ReadyState <> WebBrowserReadyState.Complete Then Return

        'Remove the event handler to avoid this code being called twice.
        RemoveHandler webBrowser1.DocumentCompleted, loginHandler

        Dim allelements As HtmlElementCollection = webBrowser1.Document.All
        For Each webpageelement As HtmlElement In allelements
            'NPI #
            If webpageelement.GetAttribute("name") = "Provider_Id" Then
                webpageelement.SetAttribute("value", "xxxxxx")
                '-- Why would you even wait in here?? There's no reason for you to wait after only changing an attribute since nothing is loaded from the internet.
            End If

            'Clicking enter to input NPI
            If webpageelement.GetAttribute("name") = "submit1" Then

                'Adding the event handler performing our next step.
                AddHandler webBrowser1.DocumentCompleted, credentialHandler

                webpageelement.InvokeMember("focus")
                webpageelement.InvokeMember("click")
            End If
        Next
    End Sub

'Add the event handler performing our first step.
AddHandler webBrowser1.DocumentCompleted, loginHandler

webBrowser1.Navigate("https://www.lamedicaid.com/sprovweb1/provider_login/provider_login.asp")

Now every time you need to wait for the document/website to be fully loaded, just declare a new lambda and add it as an event handler to DocumentCompleted:

Dim thirdStepHandler As WebBrowserDocumentCompletedEventHandler = _
    Sub(wsender As Object, we As WebBrowserDocumentCompletedEventArgs)
        'If the WebBrowser hasn't finished loading, do not continue.
        If webBrowser1.ReadyState <> WebBrowserReadyState.Complete Then Return

        'Remove the event handler to avoid this code being called twice.
        RemoveHandler webBrowser1.DocumentCompleted, thirdStepHandler

        'Your goes code here...
    End Sub


'To wait until performing the next step (be sure to do this BEFORE navigating):
AddHandler webBrowser1.DocumentCompleted, thirdStepHandler

Thank you so much for the explanation the code works wonderfully! However, I have one page where I have to select items from a drop-down menu then certain text boxes are displayed. Would you happen to know how I could go about capturing these? — Cole Perrault, Dec 05 '17 at 19:14
@ColePerrault : If my answer solved your problem please mark it as accepted by pressing the tick/check mark on the left of my post. — Visual Vincent, Dec 05 '17 at 19:29
@ColePerrault : As for your other request (which is off-topic in this question): Inject some Javascript in the page that reacts to the drop-down's [**`onchange` event**](https://www.w3schools.com/jsref/event_onchange.asp), then have that call an external VB.NET method. Here's an example of the calling a .NET method from JS (in C# but can be converted to VB.NET with an online converter): https://stackoverflow.com/a/5434220 — Visual Vincent, Dec 05 '17 at 19:30
@ColePerrault : There's also this solution: https://stackoverflow.com/a/9111151 -- or this: https://www.codeproject.com/Articles/25769/Handling-HTML-Events-from-NET-using-C — Visual Vincent, Dec 05 '17 at 19:34

webbrowser auto navigation event

1 Answers1

Linked