Accessibility considerations

[Matthias Clasen]

Hi,

I would like to discuss strategies for implementing accessibility
features in Wayland that will be needed for a fully featured desktop.
There's a whole list of individual items, but for now I'd like to
focus on keyboard accessibility features, ie roughly the feature that
traditionally labeled AccessX:

- slow keys (only accept key presses after a delay)
- sticky keys (support one-key-at-a-time operation by latching and
locking modifiers)
- bounce keys (reject quick key presses of the same key)
- toggle keys (special key bindings to enable the aforementioned
features: hold-shift-for-8-seconds for slow keys,
shift-5-times-in-a-row for sticky keys)

The following two are not technically AccessX, but are close enough to
include them:
- key repeat
- visual bell (all of the AccessX features have configurable bells;
the visual bell can be configured to be per-window or global)

Under X, all of these features are implemented in the XKB code in the
server (the visual bell gets handled by a client that selects for for
XkbBellNotify. In GNOME, that is mutter).

Wayland uses libxkbcommon to implement xkb-like keyboard handling, but
as far as I know, no Wayland compositor currently implements any of
the features I listed above. xwayland inherited the X server
implementation and thus, X clients do get AccessX features, but
Wayland clients don't. Not an ideal situation.

In GNOME, we have a client-side implementation of key repeat, which
has some issues that we recently worked around by introducing an extra
roundtrip before synthesizing repeat presses. I recently spent an
evening trying to see how far I can get in implementing AccessX
client-side (see
https://git.gnome.org/browse/gtk+/log/?h=matthiasc/wayland/slowkeys).
The results were mixed, I would say. While I have mostly working code
for all of the AccessX features, there are some limitations:

- For 'true' slow keys, the modifier keys should be delayed as well.
My client-side implementation does not do that, since the modifier
processing happens compositor-side, and I can only delay the events
that already have modifiers applied.

- I did not attempt to implement the visual bell client-side; flashing
the screen by mapping a fullscreen window temporarily would just be
too much of a hack. More recently, Jonas added a bell request to the
private gtk_shell interface between gtk+ and mutter, so we can now
reuse the existing (visual and audible) bell handling code in mutter.

- Handling the toggle keys client-side means that you can only enable
AccessX features if a client has focus; this needs to be handled in
the compositor so using one of these shortcuts can be the first thing
you do after stepping up to the computer, regardless if a client has
focus or not. We also loose a feature compared to the X session: under
X, we show a notification asking for confirmation ("Did you mean to
turn on SlowKeys?") when the toggle keys features are triggered from
the keyboard, to keep users from accidentally 'breaking' their
keyboard by resting their finger on the shift key. We can really send
notifications from deep inside a GDK backend.

- Doing this client-side implementation in GTK+ does not help one bit
for SDL or Qt or E or toytoolkit clients.

I know that one of the guiding principles of the Wayland architecure
is 'client-side', but I think the limitations listed
here are significant enough to argue for a compositor side
implementation of keyboard a11y. I don't think that this needs (a lot
of) new protocol, except for maybe a way to query and monitor state
related to these features.

One small complication with implementing AccessX compositor-side is
that we have one client that currently does it client-side: xwayland.
It would need a patch to force-disable AccessX there. This is also why
I think there should be project-wide agreement on whether these
features belong in the compositor or not. It would be really
unfortunate and bad for interoperability if we end up in a future
where these features are implemented server-side in GNOME, but
client-side in KDE or the other way around.

For the implementation itself, I guess it really belongs into
libxkbcommon, although that means it needs timers (or hooking up to a
mainloop). I find it intriguing that libxkbcommon/src/state.c has this
comment:

/*
 * This is a bastardised version of xkbActions.c from the X server which
 * does not support, for the moment:
 *   - AccessX sticky/debounce/etc (will come later)
...

Daniel, do you still have plans in this area, or did 'later' turn out
to be 'never' ?
If not, any pointers for where somebody else would start digging into this ?

Anyway, I'd like to hear people's thoughts about this topic.


Matthias