Reece,

I have to agree with you on the idea of just using UIAutomation and then using the MSAA bridge/proxy for MSAA support in the end. This leaves only one implementation in wine, requires only one to be implemented. It's likely that eventually MSAA will be phased out. Although since MSAA is already partially implemented would it be worth it to finish it up first? Then there would be support for most accessibility application while we work toward UIAutomation support.

Tying into ATK/AT-SPI or other a11y protocols so that there is native support for wine and those using it is a great idea! Although I think getting MSAA/UIAutomation is the first step, I think supporting both is a great idea. Another thought is to look at mono accessibility and it's goal of bridging UIAutomation and AT-SPI. I'm not really sure how far along they are or where they have gotten with it, but it should provide some good ideas. I don't think we should rely on it, but using it as a reference can't hurt.


-Seth


On Tue, Oct 12, 2010 at 10:55 AM, Reece Dunn <msclrhd@googlemail.com> wrote:

The current wine implementation supports GetRoleText and the like on
oleacc.dll, but does not support IAccessible (MSAA) or UIAutomation
functionality on the Win32 objects (windows, list boxes, text edit
controls, buttons, etc.).

UIAutomation consists of 4 parts:
  1/  a COM API to the accessibility functionality
  2/  UIAutomation support for all Win32 controls
  3/  an MSAA <=> UIAutomation bridge (for applications that use MSAA)
  4/  a .NET binding to the UIAutomation objects

To get MSAA working in wine, you will need to (minimally):
  1/  complete the IAccessible core support -- LresultToObject and friends
  2/  add a WM_GETOBJECT handler to all the Win32 controls and expose
IAccessible COM objects (with tests!)
  3/  implement/flesh out the WinEvents API to get the IAccessible
objects for the events

Given the move to UIAutomation, it would be easier in terms of
implementation to:
  1/  expose UIAutomation objects via the WM_GETOBJECT message
  2/  use the MSAA <=> UIAutomation bridge to support MSAA

A question here is whether the wine MSAA/WinEvents/UIAutomation
implementation should bind to the Gnome/KDE accessibility APIs
(GAIL/???). The benefit here is that the Windows applications will
integrate into the native Linux a11y technologies. The problems here
are (a) getting Alexandre's buy in to use the GAIL APIs if present and
(b) whether these are going to change in the move over to Gnome3.

In terms of applications to test, Firefox supports at least MSAA
bindings as do Qt applications. Others will to, to varying degrees
(some relying on the native Win32 support), but FF and Qt implement
their own IAccessible bindings.

In terms of testing this via applications (in addition to the
regression tests that will need to be provided), there are various
assistive technologies that make use of MSAA/UIAutomation -- JAWS,
Dragon Naturally Speaking, etc. I am not familiar with NVDA.

- Reece