1

Screenreaders access information in other programs via what I suppose is a Windows API. Can anyone comment on how this is done and provide a link to some documentation?

Are there any serious limitations I should be aware of?

If you know anything about how this works in other operating systems that would be of interest as well.

thatidiotguy
  • 8,701
  • 13
  • 60
  • 105

2 Answers2

3

Most screen readers use UI Automation to access information in other programs. Some applications don't provide accessibility information in this way (bad programmers! bad! bad!), so some screen readers use mirror drivers to intercept the low-level drawing operations, and then regenerate the application's information from there. (I don't recommend this; it's painful, prone to hang your system, and not supported in Windows 8 and above. But, if you really have to do it, well, you really have to do it.)

IOS has something similar, although I'm not familiar with it. Useful tags on stack overflow are [ui-automation] and [Microsoft-ui-automation].

Eric Brown
  • 13,774
  • 7
  • 30
  • 71
  • The spec there does not list UI Automation as available on Windows 8. Is this just an oversight? – thatidiotguy Jul 24 '13 at 17:37
  • Would this allow a program to grab highlighted text on the currently focused window? – thatidiotguy Jul 24 '13 at 17:38
  • @thatidiotguy - yes, it's just an oversight. To grab highlighted text, well, that's more complicated; see [my answer](http://stackoverflow.com/questions/4243944/how-to-get-selected-text-from-any-window-using-ui-automation-c-sharp/17604029#17604029) to that question. – Eric Brown Jul 24 '13 at 17:41
  • Hmmm so Adobe Reader AND internet explorer both do not support that huh? Any way, great answers, thanks for coming back for the comments. – thatidiotguy Jul 24 '13 at 17:45
  • IE 10+ probably supports TextPattern (I wrote that code 4 years ago, before IE went on an accessibility binge); I don't know about current versions of Adobe Reader. Essentially, I started with TextPattern, and if that failed, I had a series of fallbacks. – Eric Brown Jul 24 '13 at 17:49
1

Screen readers generally access the UI through the Microsoft Active Accessibility and/or UI Automation.

Adrian McCarthy
  • 45,555
  • 16
  • 123
  • 175
  • *Please* don't use Active Accessibility. It's slow and nasty; UI Automation will talk to Active Accessibility implementations, and do it faster than you can. MSAA is very chatty, which makes it slow when run cross-proc. UI Automation runs the MSAA interface in-proc (which is much faster), and then sends the results cross-proc. – Eric Brown Jul 24 '13 at 17:31