Accessibility considerations

[Daniel Stone]

Hi,

On 11 March 2016 at 09:31, Pekka Paalanen <ppaalanen at gmail.com> wrote:
> On Fri, 11 Mar 2016 15:46:21 +1000
> Peter Hutterer <peter.hutterer at who-t.net> wrote:
>> On Thu, Mar 10, 2016 at 01:43:13PM -0500, Matthias Clasen wrote:
>> > I would like to discuss strategies for implementing accessibility
>> > features in Wayland that will be needed for a fully featured desktop.
>> > There's a whole list of individual items, but for now I'd like to
>> > focus on keyboard accessibility features, ie roughly the feature that
>> > traditionally labeled AccessX:
>> >
>> > - slow keys (only accept key presses after a delay)
>> > - sticky keys (support one-key-at-a-time operation by latching and
>> > locking modifiers)
>> > - bounce keys (reject quick key presses of the same key)
>> > - toggle keys (special key bindings to enable the aforementioned
>> > features: hold-shift-for-8-seconds for slow keys,
>> > shift-5-times-in-a-row for sticky keys)
>> >
>> > The following two are not technically AccessX, but are close enough to
>> > include them:
>> > - key repeat
>> > - visual bell (all of the AccessX features have configurable bells;
>> > the visual bell can be configured to be per-window or global)
>> >
>> > Under X, all of these features are implemented in the XKB code in the
>> > server (the visual bell gets handled by a client that selects for for
>> > XkbBellNotify. In GNOME, that is mutter).
>>
>> [...]
>>
>> > I know that one of the guiding principles of the Wayland architecure
>> > is 'client-side', but I think the limitations listed
>> > here are significant enough to argue for a compositor side
>> > implementation of keyboard a11y. I don't think that this needs (a lot
>> > of) new protocol, except for maybe a way to query and monitor state
>> > related to these features.
>>
>> [...]
>>
>> > Anyway, I'd like to hear people's thoughts about this topic.
>>
>> fwiw, I fully agree that this need to be implemented on the compositor side.
>> Client-side first is a nice principle for things that are client-specific
>> but I see accessibility as a feature of the user, not of the software. Thus
>> it falls into the same category as other user-specific features like tapping
>> that are already implemented in the compositor (or even hidden inside
>> libinput).
>
> Yes, that sound perfectly reasonable.

Yes, I fully agree. It's not that we looked at AccessX and decided to
leave it to the toolkits, but just that it was never really done, and
I've not worked particularly much on input since.

> I wonder, what protocol interfaces would be involved?
>
> Would e.g. slow keys cause wl_keyboard key events to be delayed? If yes,
> would you alter the event timestamps to correspond the delay, or would
> they continue to be the original physical action timestamp?
>
> I have a feeling it's a bit of a vague question, since I'm not aware of
> anyone seriously using key event timestamps for anything, but I've
> raised the idea of verifying key repeat count against the time between
> key down and key up.

Altered timestamps; there isn't a lot else which makes sense,
particularly when you consider that other events could potentially
arrive in the meantime.

> Or should "normal office keyboard usage" become input method protocols
> instead of wl_keyboard, and wl_keyboard be left for "low-level" keyboard
> events?
>
> One counter use case coming to mind are games, once again, which want
> low-level keyboard events and do not want to enable e.g. slow keys or
> even pop up a question if you hold a shift down. Would there be
> protocol to disable input accessibility, or would using wl_keyboard
> already imply that? Or should it be left just as a DE user setting one
> just disables manually for everything?
>
> Should apps opt-in or opt-out of accessibility?

Opt out.

> Btw. Matthias guessed an implementation would belong in xkbcommon, but
> that is also used client-side. How does that fit in with server-side
> accessibility implementation?

That's fine; it's different parts of xkbcommon being used by the
client and the server. The client only uses the lookup of (state +
key-press/release) -> syms, whereas it's the server that actually runs
the state machinery to determine what to do. This only affects the
latter (and users of the API, e.g. you have to know to postpone
certain presses for slowkeys), so I don't really see it as having much
any effect on the protocol.

On 11 March 2016 at 22:07, Bryce Harrington <bryce at osg.samsung.com> wrote:
> Along this line of thinking, I notice that weston-simple-im.c implements
> compose sequences, but should compose key be client-side or might it fit
> in with something like this on the compositor side?  I'm a bit confused
> how compose key support is intended to work in Weston.

It's supposed to be either client-side (when using wl_keyboard), or
implemented by a server-side input method such as provided by
weston-simple-im (when using the text-based protocol). We don't have
enough information about the client state to implement it on the
server when the client uses wl_keyboard. xkbcommon also provides a
composition API for clients to use, so it's not overly difficult.

Cheers,
Daniel