[libreoffice-accessibility] Re: ESC meeting minutes: 2022-05-26 [IAccessible2 support in JAWS]

[Michael Weghorn]

Hi Marco, all,

On 09/06/2022 09.39, Marco Zehe wrote:
> AFAIK, QT exposes accessibility information to UIA on Windows. They switched over from an MSAA implementation to UIA some time in the QT5 time frame: https://www.qt.io/blog/2018/02/20/qt-5-11-brings-new-accessibility-backend-windows.

Indeed, that was what I had come across as well. (I had linked the 
corresponding Qt commit in the wiki article about adding UIA support.)

> So, using QT widgets in the VCL certainly would help with that, UIA implementation bugs in the Windows QT layer not withstanding of course. So, there are advantages of using QT on Windows where appropriate, but that also entails the danger of inheriting QT UIA bugs.

Very true. While looking a bit into a11y of the Qt-based LO VCL plugins 
on Linux, I also came across some issues that need fixing for the AT-SPI 
integration in the Qt library rather than in LO.

> I don't know anything about GTK4, so cannot make any qualified statement.

At a quick glance, everything underneath Gtk 4's "gtk/a11y" directory in 
git ( https://gitlab.gnome.org/GNOME/gtk/-/tree/main/gtk/a11y ) still 
looks AT-SPI only, though that might change at some point.

> The original advantages of IAccessible2 and GTK3 was that they were made to be very closely related to one another in terms of concepts. But with GTK4 and other frameworks becoming more prominent, this has only been in a maintained state for years, not really further developed, except for adding necessary missing pieces for new HTML widgets or markup. UIA, on the other hand, has turned out to be much more flexible especially with the latest enhancements published by Microsoft. Custom property sets etc., which allows for various annotations in MS Word, Excel etc. NVDA has a pull request for implementation of some of these newest UIA technologies into their support for MS Office here: https://github.com/nvaccess/nvda/pull/13387
> 
> So, in the long run, it is probably safest to indeed invest in switching to an UIA implementation. 
> Note, however, that you may still need to do some work yourselves for the document specific stuff for Writer, Calc, and Impress specifically, since probably not everything is available in the QT libraries that you need.

As I understand it from what I have seen so far, the concept for current 
LO a11y is also largely based on the same concepts as 
IAccessible2/AT-SPI, and Qt a11y API is also largely based on the same.

IIUC, what current platform-specific a11y integrations in LO (gtk3, 
qt5/qt6, winaccessibility) do is mostly provide a wrapper around/bridge 
between LO a11y API and the platform-specific API (ATK for gtk3 which is 
bridged to AT-SPI by ATK, Qt for qt5/qt6 which is then bridged to AT-SPI 
by the Qt library for the Linux case, MSAA/IAccessible2 for 
winaccessibility).

Most of the LO UI and document-specific a11y is implemented in a 
platform-independent way (VCL toolkit for the UI) and the mapping to the 
platform-specific implementations happens in a thin layer.

In my understanding, that would remain unchanged in case of using either 
Gtk or Qt on Windows as well. As far as a11y is concerned, this would 
essentially mean dropping the MSAA/IAccessible2 bridge contained in the 
"winaccessibility" directory and reusing the code that is also used for 
Linux (and leave the mapping to UIA to Gtk/Qt).

Should that turn out to be a reasonable approach (which I don't know!), 
my expectation would be that this would essentially give us the same set 
of features with UIA that we currently have with IAccessible2.

I suppose that supporting new UIA concepts/features in addition would 
probably require more fundamental changes than "just" adding a new 
wrapper/bridge for UIA around existing LO a11y interfaces (either a 
custom one, or using Gtk/Qt).
But I'm not an expert and have only little experience with a11y so far 
and not looked into UIA any closer, so all of the above would need 
deeper knowledge/further investigation for a more reliable statement.


Unless there are resources to work on UIA specifically, I tend to think 
it would make sense to focus on improving the existing 
IAccessible2-based implementation ("winaccessibility") (and fixing 
issues that are not platform-specific) for now if that's (still) 
sufficiently supported by AT in practice, and reconsider what to do 
about UIA at some later point in time (at which there might also be news 
on the state of gtk4 and qt5/qt6 a11y).

Happy to hear what others think about this.

Michael