0

I am starting some research on a project that involves navigating the web with speech feedback from SAPI. This application will be used by the blind and visually impaired to navigate the web with full speech feedback of current selected object and the ability to read the entire page and fill out forms. I have a mass amount of experience in SAPI and the accessibility word, however I'm not so familiar with HTML and the best approach on taking on this project and i would like to ask for guidance from someone that may have experience in this area. The basic concepts, which I've already explained a little is to enable a blind user to use this standalone product without a screen reader and have the ability to navigate the web page with a keyboard and receive speech feedback from a synthesizer i have created. I can use the Accessibility API's built into windows to display location for low vision users.

Thank you!

devHead
  • 794
  • 1
  • 15
  • 38
  • Side notes: no need to add "thank you notes" - upvote/comment/accept instead. You have not put useful tags (your question looks like "how to use accessibility with browser", while tags include "webbrowser" and "sapi"). And scope of the question may be a bit too wide for SO (feels like "how to design JAWS clone"). – Alexei Levenkov Oct 12 '12 at 17:02
  • I'm not limiting myself to IAccessible due to the fact i know there's many other ways to handle this with internal hooks to IWebBrowser2. The question is left to a wider scope on purpose, i want to get any information i can without limiting myself to any 1 methodology. I'm sure someone out there appreciates my "Thank you notes"... I'm trying to show a little appreciation for help/support... – devHead Oct 12 '12 at 17:16
  • FYI, JAWS is horrible with IE.... – devHead Oct 12 '12 at 17:17

2 Answers2

1

IE supports all accessibility interfaces (I believe all other browsers do too - sample on SO ).

There are 2 parts - code access using IAccessible and related interfaces and good HTML (i.e. correct use of semantic tags like UL/LI) markup with possible use of additional CSS accessibility aria-XXXX attributes (i.e. aria-expanded)

Possible starting link - What's New for Accessibility in IE8

Community
  • 1
  • 1
Alexei Levenkov
  • 98,904
  • 14
  • 127
  • 179
  • Yes, i'm very familiar with IAccessible. I have used it to automate windows applications, but never used it with IE. I know i can use AccessibleObjectFromPoint() to obtain text beneath cursor and also names of object/text boxes. my challenge is mainly to do this on focus change, obtain the current focused object and decipher what it is against a list of predefined controls, then figure out the proper way of obtaining that controls information and attributes. I'm diving into an area i'm unfamiliar with and haven't tested yet, just have loads of theories i haven't applied. – devHead Oct 12 '12 at 16:23
  • ideally, i would like to hook the browser, i may have to do a system wide hook in combination of using AccessibleObjectFromEvent() – devHead Oct 12 '12 at 16:26
0

I have come to the conclusion in order to get a talking web browser i actually had to hook an instance of the WebBrowser Control with Set setwindowshookex. In my setwindowshookex callback i monitor for the event: OB_FOCUS (focus changed, could be keyboard or mouse) i then pass the object instance that has focus to the AccessibleObjectFromEvent API. Next, i get the IAccessible interface to this object and then enumerate any children if present and compare them against a known object list that relates to HTML/Web objects to use the proper class that will parse this field and announce text if available, if the user is in an editable field, focused buttons, etc... I also use the IAccessible class to get the objects location and present location to the user by drawing a rectangle on the current focused object. I have also created a class that extracts just link, text, removes images and more to make for an easier, more enjoyable web experience for people with visual impairments. I still have some quarks to work out, but progress is moving forward!

devHead
  • 794
  • 1
  • 15
  • 38