wx.html2.WebView and screen readers

Question

In wxPython 4.0 Phoenix, I'm trying to use wx.html2.WebView with a screen reader.

Whether with JAWS or NVDA, I have to make a left mouse click on the Widget to be able to see my page in an accessible Web interface.

Here is my code, but know that I'm having the same problem with the LoadURL method.

Should I add something so that the focus is directly in the web interface as soon as the Widget is displayed?

Thank you in advance for your answers.

import wx
import wx.html2

class MyWebView (wx.Dialog):

    def __init__(self, parent):
        wx.Dialog.__init__(self, parent)
        sizer = wx.BoxSizer(wx.VERTICAL)
        self.browser = wx.html2.WebView.New(self)
        self.changeBtn = wx.Button (self, -1, label="Change page")

        # The pages that I want to display disposed in a tuple.
        self.pages = (
        "<html><head><title>Hello everyone !</title></head><body><h1>We're testing wx.html2.WebView with a Screen Reader !</h1></body></html>",
        "<html><head><title>Second page !</title></head><body><h1>This is a second page</h1></body></html>",
        "<html><head><title>Third page !</title></head><body><h1>This is a third page</h1></body></html>"
        )

        self.index = 0
        self.display = self.pages[self.index]
        sizer.Add(self.browser, 1, wx.EXPAND, 10)
        sizer.Add(self.changeBtn)
        self.SetSizer(sizer)
        self.SetSize((700, 700))

    # Events.
        self.changeBtn.Bind (wx.EVT_BUTTON, self.onChangePage)

    def onChangePage (self, evt):
        self.index += 1
        if self.index == len (self.pages):
            self.index = 0
        self.display = self.pages[self.index]
        self.browser.SetPage(self.display, "")

if __name__ == '__main__':
    app = wx.App()
    dialog = MyWebView (None)
    dialog.browser.SetPage(dialog.display, "")
    dialog.Show ()
    app.MainLoop()

Kind regards.

score 2 · Accepted Answer · answered Aug 14 '19 at 20:38

Under windows, due to a strange bug with Internet explorer Server, you effectively need to click somewhere in the widget so that the screen readers can interact with it. Once that initial click has been made, screen readers perfectly work with the widget.

Note that the same problem still occurs with the latest wxWidgets 3.1.3 in C++. In fact, nor wxWidgets, nor wxPython are responsible for this bug. It also occurs when embedding Internet Explorer Server directly using ActiveX and Win32 API.

You can generate the required initial mouse click so that the widget becomes accessible with something like this:

robot = wx.UIActionSimulator() 
self.browser.SetFocus() 
position = self.browser.GetPosition() 
position = self.browser.ClientToScreen(position) 
robot.MouseMove(position) 
robot.MouseClick()

wx.UIActionSimulator is a class allowing to generate user events like a robot. Normally it is used for demos, UI automated tests, or replaying recorded macros. Here we use it to make the desired mouse click. A similar solution also works in C++.

We set the focus to the widget
We get its top-left position relative to the window
The UIActionSimulator requires absolute screen coordinates, so we need to convert the position
We move the mouse and click

By clicking on the very top-left-most point of the widget, it's very unlikely to produce an undesired effect, while not totally impossible.

score 0 · Answer 2 · answered Aug 27 '22 at 17:14

As already suggested in the first answer to this question, a workaround for this issue might be an automation of mouse click somewher to the HTML page, such as to its top left corner. However, in my case the code proposed in the first answer didn't work. To ensure this workaround is effective, the click should be performed after the page has been successfully loaded, that is, after the wx.html2.EVT_WEBVIEW_LOADED event bound to the WebView widget is fired.

A plain dialog with the HTML WebView widget employing the fix then might look like this:

import wx
import wx.html2

class HtmlDialog(wx.Dialog):

    def __init__(self, title, parent=None):
        super(HtmlDialog, self).__init__(parent=parent, title=title, size=(1000, 800))

        self.addWidgets()
        self.Centre()
        self.ShowModal()

    def addWidgets(self):
        self.panel = wx.Panel(self)
        vbox = wx.BoxSizer(wx.VERTICAL)

        self.browser = wx.html2.WebView.New(self.panel)
        self.browser.Bind(wx.html2.EVT_WEBVIEW_LOADED, self.clickToPage)
        html = "<p>Hello there! <a href='#'>Click me</a>.</p>"
        self.browser.SetPage(html, "")
        vbox.Add(self.browser, 1, wx.EXPAND | wx.ALL, 5)

        self.panel.SetSizer(vbox)

    def clickToPage(self, event):
        robot = wx.UIActionSimulator()
        position = self.browser.GetPosition()
        position = self.browser.ClientToScreen(position)
        robot.MouseMove(position)
        robot.MouseClick()
        self.browser.SetFocus()

wx.html2.WebView and screen readers

2 Answers2

Linked