How do I capture text from a HTML element given its id="" from a WebView2 in VB.NET?

Question

I am trying to read a table and get the fields i need from that table with Webview2.

I am able to get the source code of the webpage but I'm stumped beyond that. The element show up as shown in the picture. but they are loaded in AFTER. So they dont exist on the address im trying to parse.

So if i CTRL SHIFT C, I can see the elements there. But if i view the source of the HTML, they are not there. So when i try to pull the data, The elements are not there, and it returns null.

When i save the webpage locally and inspect the main DATA.HTML file the elements are not on that main file. They are on a html file called default.HTML that is in the sources folder that was saved with the DATA.HTML. I am able to pull the elements from that nested default.html file when its saved to my computer, but NOT the DATA.HTML file.

I understand this is hard to answer when i cant show the source of the HTML i need to get. Can someone point me in the right direction as to how to find the default.html file that data.html is reading in a live environment?

Dim firstNameText As String = Await WebView21.ExecuteScriptAsync("document.getElementById('m.first_name').textContent");

MessageBox.Show( "First name: """ & firstNameText & """." )

Answer 1

You have wrong event handler signature here:

Private Async Function WebView2_NavigationCompletedAsync(
                sender As Object, e As CoreWebView2NavigationCompletedEventArgs) _
                As Task Handles WebView21.NavigationCompleted
    ' ...
End Function

an event handler is a Sub/void not a Function and does not return any value of any type.

The correct signature is:

Private Sub WebView2_NavigationCompletedAsync(
                    sender As Object, e As CoreWebView2NavigationCompletedEventArgs) _
                    Handles WebView21.NavigationCompleted
    ' ...
End Sub

As for the webView2 part, make the handle Async method and get the content of the target td as follows:

Private Async Sub WebView2_NavigationCompletedAsync(
                    sender As Object,
                    e As CoreWebView2NavigationCompletedEventArgs) _
                    Handles WebView21.NavigationCompleted
    Dim firstName = (Await WebView21.
        ExecuteScriptAsync("document.getElementById('m.first_name').textContent;")).
        Trim(ChrW(34))

    Debug.WriteLine(firstName)
End Sub

You could try the querySelector() method as well:

Private Async Sub WebView2_NavigationCompletedAsync(
                    sender As Object,
                    e As CoreWebView2NavigationCompletedEventArgs) _
                    Handles WebView21.NavigationCompleted
    Dim firstName = (Await WebView21.
        ExecuteScriptAsync("document.querySelector('#m\\.first_name').textContent;")).
        Trim(ChrW(34))

    Debug.WriteLine(firstName)
End Sub

Answer 2

You haven't provided enough code to know exactly what the issue is.

According to WebView2.CoreWebView2 Property:

Accesses the complete functionality of the underlying CoreWebView2 COM API. Returns null until initialization has completed. See the WebView2 class documentation for an initialization overview.

It's not clear how you're initializing CoreWebView2. The issue may be with your CoreWebView2 initialization and order of execution. You can use Debug.WriteLine to confirm this. To help debug the issue, subscribe to the following events:

WebView2 events:

• CoreWebView2InitializationCompleted
• NavigationCompleted

CoreWebView2 events:

• DOMContentLoaded

---

Below shows how to set the UserDataFolder for both explicit and implicit initialization.

Create a Windows Forms App (.NET Framework)

Download / install NuGet package: Microsoft.Web.WebView2 (v 1.0.1293.44)

Note: WebView2 version 1.0.1343.22 seems to have a bug that causes a null reference exception. This can be seen by placing the following code within the CoreWebView2InitializationCompleted event handler:

   Private Sub WebView21_CoreWebView2InitializationCompleted(sender As Object, e As CoreWebView2InitializationCompletedEventArgs) Handles WebView21.CoreWebView2InitializationCompleted
       Dim wv As WebView2 = DirectCast(sender, WebView2)

       Debug.WriteLine($"UserDataFolder: {wv.CoreWebView2.Environment.UserDataFolder}")
       Debug.WriteLine($"Edge Browser version: {wv.CoreWebView2.Environment.BrowserVersionString}")
   End Sub

However, explicit initialization, using CoreWebView2Environment as shown below, seems to work in WebView2 version 1.0.1343.22.

On the form, I've used TableLayoutPanel, TextBox (name: textBoxAddressBar), Button (names: btnBack, btnForward, btnGo), and WebView2 (name: WebView21) controls.

Here's what the form looks like:

In the code below, each of the options contains some common code. To avoid confusion, I've included the complete code (for each of the options) and added explanations (as comments) within the code. Each of the options below have been tested.

Option 1 - explicit initialization (CoreWebView2Environment)

Imports System.IO
Imports Microsoft.Web.WebView2.Core
Imports Microsoft.Web.WebView2.WinForms

Public Class Form1
    Private Async Sub Form1_Load(sender As Object, e As EventArgs) Handles MyBase.Load

        LogMsg($"WebView2 version: {GetWebView2Version()}")

        'explicitly initialize CoreWebView2
        Await InitializeCoreWebView2Async(WebView21)

        'since we've used explicit initialization, which is Awaited,
        'if desired, one can subscribe to CoreWebView2 events here
        'instead of within CoreWebView2InitializationCompleted
        'subscribe to events
        'AddHandler WebView21.CoreWebView2.DOMContentLoaded, AddressOf CoreWebView2_DOMContentLoaded
        'AddHandler WebView21.CoreWebView2.HistoryChanged, AddressOf CoreWebView2_HistoryChanged

        LogMsg($"before setting source")

        'ToDo: update with desired URL
        'after setting Source property execution continues immediately
        WebView21.Source = New Uri("http://127.0.0.1:9009/index.html")

        LogMsg($"after setting source")
    End Sub

    Public Function GetWebView2Version() As String
        Dim webView2Assembly As System.Reflection.Assembly = GetType(WebView2).Assembly
        Return FileVersionInfo.GetVersionInfo(webView2Assembly.Location).ProductVersion
    End Function

    Public Async Function InitializeCoreWebView2Async(wv As WebView2, Optional userDataFolder As String = Nothing) As Task
        Dim options As CoreWebView2EnvironmentOptions = Nothing
        Dim webView2Environment As CoreWebView2Environment = Nothing

        If String.IsNullOrEmpty(userDataFolder) Then
            'create unique name for web cache folder in temp folder
            'userDataFolder = System.IO.Path.Combine(System.IO.Path.GetTempPath(), System.Guid.NewGuid().ToString("N"))
            userDataFolder = System.IO.Path.Combine(System.IO.Path.GetTempPath(), Path.GetFileNameWithoutExtension(System.Reflection.Assembly.GetExecutingAssembly().Location))
        End If

        'webView2Environment = await CoreWebView2Environment.CreateAsync(@"C:\Program Files (x86)\Microsoft\Edge\Application\105.0.1343.50", userDataFolder, options);
        webView2Environment = Await CoreWebView2Environment.CreateAsync(Nothing, userDataFolder, options)

        LogMsg("before EnsureCoreWebView2Async")

        'wait for CoreWebView2 initialization
        Await wv.EnsureCoreWebView2Async(webView2Environment)

        LogMsg("after EnsureCoreWebView2Aync")

        LogMsg("UserDataFolder folder set to: " & userDataFolder)
    End Function

    Private Sub LogMsg(ByVal msg As String)
        msg = String.Format("{0} {1}", DateTime.Now.ToString("yyyy/MM/dd HH:mm:ss:fff"), msg)
        Debug.WriteLine(msg)
    End Sub

    Public Sub WebsiteNavigate(ByVal wv As WebView2, ByVal dest As String)

        If Not wv Is Nothing AndAlso Not wv.CoreWebView2 Is Nothing Then
            If Not String.IsNullOrEmpty(dest) Then

                If Not dest = "about:blank" AndAlso
                   Not dest.StartsWith("edge://") AndAlso
                   Not dest.StartsWith("file://") AndAlso
                   Not dest.StartsWith("http://") AndAlso
                   Not dest.StartsWith("https://") AndAlso
                   Not System.Text.RegularExpressions.Regex.IsMatch(dest, "^([A-Z]|[a-z]):") Then

                    'URL must start with one of the specified strings
                    'if Not, pre-pend with "http://"
                    'Debug.Print("Prepending ""http://"" to URL.")

                    'set value
                    dest = "http://" & dest
                End If

                'option 1
                wv.Source = New Uri(dest, UriKind.Absolute)

                'option 2
                'wv.CoreWebView2.Navigate(dest)

            End If
        End If
    End Sub

    Private Sub textBoxAddressBar_KeyDown(sender As Object, e As KeyEventArgs) Handles textBoxAddressBar.KeyDown
        If e.KeyCode = Keys.Enter AndAlso WebView21 IsNot Nothing Then
            WebsiteNavigate(WebView21, textBoxAddressBar.Text)
        End If
    End Sub

    Private Sub btnGo_Click(sender As Object, e As EventArgs) Handles btnGo.Click
        WebsiteNavigate(WebView21, textBoxAddressBar.Text)
    End Sub

    Private Async Sub CoreWebView2_DOMContentLoaded(sender As Object, e As CoreWebView2DOMContentLoadedEventArgs)
        LogMsg($"CoreWebView2_DOMContentLoaded")

        Dim cwv2 As CoreWebView2 = DirectCast(sender, CoreWebView2)

        Try
            Dim result As String = Await cwv2.ExecuteScriptAsync("document.getElementById('m.first_name').textContent")
            Debug.WriteLine($"result: {result}")
        Catch ex As AggregateException
            'ToDo: change code as desired

            LogMsg($"Error: {ex.Message}")

            If ex.InnerExceptions IsNot Nothing Then
                For Each ex2 As Exception In ex.InnerExceptions
                    LogMsg($"{ex2.Message}")
                Next
            End If

            LogMsg($"StackTrace: {ex.StackTrace}")
        End Try

    End Sub

    Private Sub CoreWebView2_HistoryChanged(sender As Object, e As Object)
        Dim cwv2 As CoreWebView2 = DirectCast(sender, CoreWebView2)
        btnBack.Enabled = WebView21.CoreWebView2.CanGoBack
        btnForward.Enabled = WebView21.CoreWebView2.CanGoForward

        'update address bar
        textBoxAddressBar.Text = cwv2.Source
        textBoxAddressBar.Select(textBoxAddressBar.Text.Length, 0)
    End Sub

    Private Sub WebView21_CoreWebView2InitializationCompleted(sender As Object, e As CoreWebView2InitializationCompletedEventArgs) Handles WebView21.CoreWebView2InitializationCompleted
        Dim wv As WebView2 = DirectCast(sender, WebView2)

        LogMsg($"WebView21_CoreWebView2InitializationCompleted")
        LogMsg($"UserDataFolder: {WebView21.CoreWebView2.Environment.UserDataFolder}")
        LogMsg($"Edge Browser version: {WebView21.CoreWebView2.Environment.BrowserVersionString}")

        'subscribe to events
        AddHandler wv.CoreWebView2.DOMContentLoaded, AddressOf CoreWebView2_DOMContentLoaded
        AddHandler wv.CoreWebView2.HistoryChanged, AddressOf CoreWebView2_HistoryChanged
    End Sub

    Private Sub WebView21_NavigationCompleted(sender As Object, e As CoreWebView2NavigationCompletedEventArgs) Handles WebView21.NavigationCompleted
        LogMsg($"WebView21_NavigationCompleted")
    End Sub
End Class

Option 2 - explicit initialization (CreationProperties)

Imports System.IO
Imports Microsoft.Web.WebView2.Core
Imports Microsoft.Web.WebView2.WinForms

Public Class Form1
    Private Async Sub Form1_Load(sender As Object, e As EventArgs) Handles MyBase.Load

        LogMsg($"WebView2 version: {GetWebView2Version()}")

        'set UserDataFolder
        Dim userDataFolder As String = Path.Combine(Path.GetTempPath(), Path.GetFileNameWithoutExtension(System.Reflection.Assembly.GetExecutingAssembly().Location))
        WebView21.CreationProperties = New CoreWebView2CreationProperties() With {.UserDataFolder = userDataFolder}

        'explicitly initialize CoreWebView2
        Await WebView21.EnsureCoreWebView2Async()

        'since we've used explicit initialization, which is Awaited,
        'if desired, one can subscribe to CoreWebView2 events here
        'instead of within CoreWebView2InitializationCompleted
        'subscribe to events
        'AddHandler WebView21.CoreWebView2.DOMContentLoaded, AddressOf CoreWebView2_DOMContentLoaded
        'AddHandler WebView21.CoreWebView2.HistoryChanged, AddressOf CoreWebView2_HistoryChanged

        LogMsg($"before setting source")

        'ToDo: update with desired URL
        'after setting Source property execution continues immediately
        WebView21.Source = New Uri("http://127.0.0.1:9009/index.html")

        LogMsg($"after setting source")
    End Sub

    Public Function GetWebView2Version() As String
        Dim webView2Assembly As System.Reflection.Assembly = GetType(WebView2).Assembly
        Return FileVersionInfo.GetVersionInfo(webView2Assembly.Location).ProductVersion
    End Function

    Private Sub LogMsg(ByVal msg As String)
        msg = String.Format("{0} {1}", DateTime.Now.ToString("yyyy/MM/dd HH:mm:ss:fff"), msg)
        Debug.WriteLine(msg)
    End Sub

    Public Sub WebsiteNavigate(ByVal wv As WebView2, ByVal dest As String)

        If Not wv Is Nothing AndAlso Not wv.CoreWebView2 Is Nothing Then
            If Not String.IsNullOrEmpty(dest) Then

                If Not dest = "about:blank" AndAlso
                   Not dest.StartsWith("edge://") AndAlso
                   Not dest.StartsWith("file://") AndAlso
                   Not dest.StartsWith("http://") AndAlso
                   Not dest.StartsWith("https://") AndAlso
                   Not System.Text.RegularExpressions.Regex.IsMatch(dest, "^([A-Z]|[a-z]):") Then

                    'URL must start with one of the specified strings
                    'if Not, pre-pend with "http://"
                    'Debug.Print("Prepending ""http://"" to URL.")

                    'set value
                    dest = "http://" & dest
                End If

                'option 1
                wv.Source = New Uri(dest, UriKind.Absolute)

                'option 2
                'wv.CoreWebView2.Navigate(dest)

            End If
        End If
    End Sub

    Private Sub textBoxAddressBar_KeyDown(sender As Object, e As KeyEventArgs) Handles textBoxAddressBar.KeyDown
        If e.KeyCode = Keys.Enter AndAlso WebView21 IsNot Nothing Then
            WebsiteNavigate(WebView21, textBoxAddressBar.Text)
        End If
    End Sub

    Private Sub btnGo_Click(sender As Object, e As EventArgs) Handles btnGo.Click
        WebsiteNavigate(WebView21, textBoxAddressBar.Text)
    End Sub

    Private Async Sub CoreWebView2_DOMContentLoaded(sender As Object, e As CoreWebView2DOMContentLoadedEventArgs)
        LogMsg($"CoreWebView2_DOMContentLoaded")

        Dim cwv2 As CoreWebView2 = DirectCast(sender, CoreWebView2)

        Try
            Dim result As String = Await cwv2.ExecuteScriptAsync("document.getElementById('m.first_name').textContent")
            Debug.WriteLine($"result: {result}")
        Catch ex As AggregateException
            'ToDo: change code as desired

            LogMsg($"Error: {ex.Message}")

            If ex.InnerExceptions IsNot Nothing Then
                For Each ex2 As Exception In ex.InnerExceptions
                    LogMsg($"{ex2.Message}")
                Next
            End If

            LogMsg($"StackTrace: {ex.StackTrace}")
        End Try
    End Sub

    Private Sub CoreWebView2_HistoryChanged(sender As Object, e As Object)
        Dim cwv2 As CoreWebView2 = DirectCast(sender, CoreWebView2)
        btnBack.Enabled = WebView21.CoreWebView2.CanGoBack
        btnForward.Enabled = WebView21.CoreWebView2.CanGoForward

        'update address bar
        textBoxAddressBar.Text = cwv2.Source
        textBoxAddressBar.Select(textBoxAddressBar.Text.Length, 0)
    End Sub

    Private Sub WebView21_CoreWebView2InitializationCompleted(sender As Object, e As CoreWebView2InitializationCompletedEventArgs) Handles WebView21.CoreWebView2InitializationCompleted
        Dim wv As WebView2 = DirectCast(sender, WebView2)

        LogMsg($"WebView21_CoreWebView2InitializationCompleted")
        LogMsg($"UserDataFolder: {WebView21.CoreWebView2.Environment.UserDataFolder}")
        LogMsg($"Edge Browser version: {WebView21.CoreWebView2.Environment.BrowserVersionString}")

        'subscribe to events
        AddHandler wv.CoreWebView2.DOMContentLoaded, AddressOf CoreWebView2_DOMContentLoaded
        AddHandler wv.CoreWebView2.HistoryChanged, AddressOf CoreWebView2_HistoryChanged
    End Sub

    Private Sub WebView21_NavigationCompleted(sender As Object, e As CoreWebView2NavigationCompletedEventArgs) Handles WebView21.NavigationCompleted
        LogMsg($"WebView21_NavigationCompleted")
    End Sub
End Class

Option 3 - implicit initialization (CreationProperties)

Imports System.IO
Imports Microsoft.Web.WebView2.Core
Imports Microsoft.Web.WebView2.WinForms

Public Class Form1
    Private Async Sub Form1_Load(sender As Object, e As EventArgs) Handles MyBase.Load

        LogMsg($"WebView2 version: {GetWebView2Version()}")

        'set UserDataFolder
        Dim userDataFolder As String = Path.Combine(Path.GetTempPath(), Path.GetFileNameWithoutExtension(System.Reflection.Assembly.GetExecutingAssembly().Location))
        WebView21.CreationProperties = New CoreWebView2CreationProperties() With {.UserDataFolder = userDataFolder}

        LogMsg($"before setting source")

        'CoreWebView2 will be implicitly initialized when
        'Source property is set
        'this doesn't wait for CoreWebView2 intialization to complete
        'so any code that exists after this statement may execute 
        'prior to CoreWebView2 intialization completing
        WebView21.Source = New Uri("http://127.0.0.1:9009/index.html")

        LogMsg($"after setting source")
    End Sub

    Public Function GetWebView2Version() As String
        Dim webView2Assembly As System.Reflection.Assembly = GetType(WebView2).Assembly
        Return FileVersionInfo.GetVersionInfo(webView2Assembly.Location).ProductVersion
    End Function

    Private Sub LogMsg(ByVal msg As String)
        msg = String.Format("{0} {1}", DateTime.Now.ToString("yyyy/MM/dd HH:mm:ss:fff"), msg)
        Debug.WriteLine(msg)
    End Sub

    Public Sub WebsiteNavigate(ByVal wv As WebView2, ByVal dest As String)

        If Not wv Is Nothing AndAlso Not wv.CoreWebView2 Is Nothing Then
            If Not String.IsNullOrEmpty(dest) Then

                If Not dest = "about:blank" AndAlso
                   Not dest.StartsWith("edge://") AndAlso
                   Not dest.StartsWith("file://") AndAlso
                   Not dest.StartsWith("http://") AndAlso
                   Not dest.StartsWith("https://") AndAlso
                   Not System.Text.RegularExpressions.Regex.IsMatch(dest, "^([A-Z]|[a-z]):") Then

                    'URL must start with one of the specified strings
                    'if Not, pre-pend with "http://"
                    'Debug.Print("Prepending ""http://"" to URL.")

                    'set value
                    dest = "http://" & dest
                End If

                'option 1
                wv.Source = New Uri(dest, UriKind.Absolute)

                'option 2
                'wv.CoreWebView2.Navigate(dest)

            End If
        End If
    End Sub

    Private Sub textBoxAddressBar_KeyDown(sender As Object, e As KeyEventArgs) Handles textBoxAddressBar.KeyDown
        If e.KeyCode = Keys.Enter AndAlso WebView21 IsNot Nothing Then
            WebsiteNavigate(WebView21, textBoxAddressBar.Text)
        End If
    End Sub

    Private Sub btnGo_Click(sender As Object, e As EventArgs) Handles btnGo.Click
        WebsiteNavigate(WebView21, textBoxAddressBar.Text)
    End Sub

    Private Async Sub CoreWebView2_DOMContentLoaded(sender As Object, e As CoreWebView2DOMContentLoadedEventArgs)
        LogMsg($"CoreWebView2_DOMContentLoaded")

        Dim cwv2 As CoreWebView2 = DirectCast(sender, CoreWebView2)

        Try
            Dim result As String = Await cwv2.ExecuteScriptAsync("document.getElementById('m.first_name').textContent")
            Debug.WriteLine($"result: {result}")
        Catch ex As AggregateException
            'ToDo: change code as desired
            LogMsg($"Error: {ex.Message}")

            If ex.InnerExceptions IsNot Nothing Then
                For Each ex2 As Exception In ex.InnerExceptions
                    LogMsg($"{ex2.Message}")
                Next
            End If

            LogMsg($"StackTrace: {ex.StackTrace}")
        End Try
    End Sub

    Private Sub CoreWebView2_HistoryChanged(sender As Object, e As Object)
        Dim cwv2 As CoreWebView2 = DirectCast(sender, CoreWebView2)
        btnBack.Enabled = WebView21.CoreWebView2.CanGoBack
        btnForward.Enabled = WebView21.CoreWebView2.CanGoForward

        'update address bar
        textBoxAddressBar.Text = cwv2.Source
        textBoxAddressBar.Select(textBoxAddressBar.Text.Length, 0)
    End Sub

    Private Sub WebView21_CoreWebView2InitializationCompleted(sender As Object, e As CoreWebView2InitializationCompletedEventArgs) Handles WebView21.CoreWebView2InitializationCompleted
        Dim wv As WebView2 = DirectCast(sender, WebView2)

        LogMsg($"WebView21_CoreWebView2InitializationCompleted")
        LogMsg($"UserDataFolder: {WebView21.CoreWebView2.Environment.UserDataFolder}")
        LogMsg($"Edge Browser version: {WebView21.CoreWebView2.Environment.BrowserVersionString}")

        'subscribe to events
        AddHandler wv.CoreWebView2.DOMContentLoaded, AddressOf CoreWebView2_DOMContentLoaded
        AddHandler wv.CoreWebView2.HistoryChanged, AddressOf CoreWebView2_HistoryChanged
    End Sub

    Private Sub WebView21_NavigationCompleted(sender As Object, e As CoreWebView2NavigationCompletedEventArgs) Handles WebView21.NavigationCompleted
        LogMsg($"WebView21_NavigationCompleted")
    End Sub
End Class

---

Here's the HTML that I used for testing:

index.html

<html>
  <head>
  </head>
  <body>
    <div id="view_m" style="display: block;">
      <div id="form_small_left">
      <table id="view_m_1" style="display: block;">
        <tbody>
          <tr>
            <th>First name:</th>
            <td id="m.first_name">Margeaet</td>
          </tr>
          <tr>
            <th>Last name:</th>
            <td id="m.last_name">Bill</td>
          </tr>
        </tbody>
      </div>
    </div>
  </body>
</html>

---

Resources

• WebView2
• CoreWebView2
• CoreWebView2Environment
• CreationProperties
• EnsureCoreWebView2Async

Answer 3

1. Use document.getElementById to get a reference to a DOM element.
2. Then use textContent or innerText to get the text (not HTML) of that element (and all its descendants) stitched together.
   • Use textContent to get text from all descendant elements, including hidden and <script> elements.
   • Use innerText to filter-out hidden elements and non-human-readable elements.
3. As you cannot directly interact with the DOM in WebView2 you will need to do it all in JavaScript inside ExecuteScriptAsync. The result of the last expression inside the script will be converted to a .NET String value and returned via the Task<String> which you can await.

Like so:

Private Async Function WebView2_NavigationCompletedAsync( ... ) As Task
    Handles WebView21.NavigationCompleted

    '''''

    Dim firstNameText As String = Await WebView21.ExecuteScriptAsync("document.getElementById('m.first_name').textContent");

    MessageBox.Show( "First name: """ & firstNameText & """." )

End Function