[a11y] LibreOffice Calc exposes 2^31 children, freezes on `GetChildren`

[Michael Weghorn]

Hi Michael,

On 2024-06-13 14:49, Michael Meeks wrote:
>> AT-SPI's flows-from and flows-to relations (and ARIA's aria-flowto) 
>> seem somewhat similar to the UIA Navigation API you mention.
> 
>      =) Ultimately we dynamically create peers as these methods are 
> called currently I imagine.

Yes, but that currently has issues, s. the previously mentioned tdf#96492.

> 
>> If they allow consistent access to off-screen content (related: 
>> tdf#96492), they could potentially be used to retrieve the 
>> previous/next heading,...
> 
>      Sure; I guess the MS APIs have the problem that the interface 
> implemented tends also to be the protocol for remote COM querying of 
> peers whereas in Linux we can cut/de-couple that and can do better at 
> least in theory.

IIUC, we'd still have a AT-SPI bus roundtrip for each 
flows-from/flows-to call, so using exclusively flows-from/flows-to to 
navigate through a larger document comes with a cost.
On the other hand, IIUC, when using AT-SPI Collection, the bus overhead 
would be lower as it's just one call to the interface method, then the 
filtering/... happens on the application side and the result is returned 
in a single D-Bus reply again. (I didn't have to do with it in practice 
yet, though.)

>> E.g. it would seem odd to me if an AT starts scrolling through the 
>> document if a "go to next heading/list item" navigation command is 
>> triggered, and then e.g. goes back if it doesn't find anything, 
>> because it can't otherwise access the previously off-screen content to 
>> search for the item.
> 
>      I guess; but I really expect that there are keybindings and/or well 
> known actions for expert users that are used left and right, and that in 
> reality tracking the focused peer and interrogating it is some 
> overwhelming majority of the use cases to the point that first making 
> that piece really, really good, fast & complete is far more important 
> than anything else; but perhaps I'm mistaken.

I agree that proper handling of focus is essential and having that work 
properly is definitely a priority.

Of course there are issues and a plan to look into those, but my 
impression this far is that reporting focus in Writer works decently *in 
general*.
If you're aware of particularly pressing issues here, please don't 
hesitate to mention them (ideally directly in Bugzilla).

I don't think focus tracking by itself is sufficient for a really good 
a11y experience, though. When it comes to navigating through documents, 
some kind of "browse mode" that allows to navigate between objects in 
documents provides an experience that users are familiar with from e.g. 
web browsers or other office suites and is something that users have 
been asking for repeatedly. (LO's own navigator provides some similar 
functionality, but less well integrated into the screen reader.)

>      From a prioritization perspective; I'd really suggest working on 
> the majority platforms for the impaired: Windows/NVDA, 

Personal preferences aside, Windows/NVDA as the most widely used 
platform indeed generally has some priority for me, as does Writer over 
Calc over everything else.

There are other factors I also take into account, though, e.g. 
involvement/contributions from others like people working in certain 
areas, user requests/tickets, possibilities to cooperate (e.g. the Orca 
maintainer reworking Orca's LibreOffice support and providing a lot of 
helpful feedback and input) or productivity (my productivity on Linux is 
way higher than on Windows, so my take is that putting some extra 
initial effort in order to be able to do most of the analysis for issues 
*also* affecting Windows on Linux usually pays off, in particular since 
the platform APIs IAccessible/AT-SPI2 are fairly similar).

> and vast-majority use-cases: of getting really good & complete API and feature coverage on 
> the focused widget, before moving off into the more tricky stuff :-)

As mentioned above: If you're aware of major issues regarding support 
for APIs use cases for the focused widget, please don't hesitate to 
mention them so they can be taken into account.

>>>      But now I shut up ;-) we're working on the web side of this; 
>>> caching bits in the browser and adding another protocol latency there 
>>> - and I'm sure we want to be handling a reasonably bounded set of 
>>> data there =)
>>
>> Is there an easy way to test COOL a11y web and impacts of potential 
>> changes?
> 
>      Ah - so; we tend to focus on the focused widget and things 'near' 
> it - adjacent table cells etc. when populating our shadow DOM. But at 
> some level the use-case we have for the a11y APIs is not really 
> different than an AT would use I think.
> 
>> (I just opened a sample Writer doc on nextcloud.documentfoundation.org 
>> and couldn't find the doc content via Accerciser in a quick test, but 
>> am also not very familiar with web content/browser a11y.)
> 
>      You will want:
> 
>          <enable type="bool" desc="Controls whether accessibility 
> support should be enabled or not." default="false">false</enable>
> 
>      Enabled in coolwsd.xml - and then to turn on screen-reading support.
> 
>      =)

Thanks, will try that!

>      Again; I'd respectfully suggest that creating APIs that make it 
> possible to easily do things that then scale badly ultimately does a 
> dis-service to the impaired; people quickly use them and write poorly 
> performing ATs.
> 
>      A nice API for navigation and/or pre-fetching to enable linear 
> reading through documents, and/or reading of headings etc. seems to me 
> far more useful (and likely to perform well) - than an API that allows 
> pre-fetch of potentially hundreds of thousands of peers - even if we 
> don't think they will change in readonly mode =)
> 
>      I think AT authors will always want all of the state of the app 
> exposed, and indeed they will want it all cached locally if they can get 
> it: but ... probably what is really useful is providing a way to write 
> simple, reliable, performant, context-aware, maintainable ATs easily - 
> and IMHO the "suck all state down as a first step" thing is not that :-)

Having at least some experience working together with AT developers 
(particularly Orca and NVDA), I'm a bit more optimistic about being able 
to find good solutions together here than the above initially sounds to 
me, in particular when thinking about an opt-in feature that doesn't 
just get used "automatically" whenever any AT is active.

But of course, I fully agree that identifying suitable APIs/interaction 
should not be neglected as part of the process.


As a side note, my expectation is that that apart from identifying the 
right way to somehow make the whole document available to ATs, many 
aspects (or current LO shortcomings) that will show up in an effort to 
implement browse mode will be relevant independent of the exact API used 
to expose off-screen content, and addressing those issues will also be 
relevant for interacting with on-screen content (e.g. 
incomplete/incorrect support for specific a11y interfaces, as one 
example you mentioned earlier).

Regards,
Michael
-------------- next part --------------
A non-text attachment was scrubbed...
Name: OpenPGP_signature.asc
Type: application/pgp-signature
Size: 840 bytes
Desc: OpenPGP digital signature
URL: <https://lists.freedesktop.org/archives/libreoffice/attachments/20240614/40e2d485/attachment.sig>