Multitouch protocol and misbehaving clients

[Chase Douglas]

On 08/31/2011 11:06 AM, Carlos Garnacho wrote:
> Hello,
> 
> I've been following the latest changes to the XInput protocol for
> multitouch and doing practical experiments with the Ubuntu shipped XI2.1
> on Mutter/Gnome-shell.

Great!

> Overall, the protocol description looks quite polished now, although I'm
> quite concerned about the strict dependence of the server on grabbing
> clients calling XIAllowTouchEvents(), and how it has no way to handle
> misbehaving clients.
> 
> Here are a few points I think it'd be good to restrain and clarify:
> 
>       * What happens if a client fails to accept/reject a touch
>         sequence?

Nothing happens until the client either: fails to accept/reject a touch
or disappears (in which case the touches are rejected).

>       * What happens if a missed, physically finished touch sequence
>         owned the pointer emulation? should pointer emulation be carried
>         over to a newly started touchpoint? The XI2proto changes mention
>         that pointer emulation is "implementation defined", so this
>         might fall into such implementation details.

The prototype server in Ubuntu would essentially hold the pointer hostage.

> Perhaps these two issues should just be dealt with as stale active
> grabs, and release the touch sequences as soon as the client that's
> stuck with these disappears. 
> 
> However, the unbounded amount of stale resources the server could create
> for unfinished touch sequences until the client quits is somewhat
> concerning.

Just to clarify here, the server may bound the amount of resources given
for touch handling. For example, in the prototype server in Ubuntu there
is a ring buffer of touch events for each touch sequence. If you
overflow the ring buffer, older touch update events are discarded (the
begin event is kept so future clients will know where the touch started
at least). This is detailed in the spec, that touch update events may be
dropped.

> And in that situation, just transferring ownership onto the
> next client not listening to unowned events might mean lots of old, out
> of context touches that would be misinterpreted as "recent enough" when
> these are replayed.

This can be an issue no matter what you do. However, we have the same
issue through other means too. If you have a machine that is really
loaded and swapping constantly, the desktop may hang for a bit. We could
throw our hands up and say the user should have more ram, or it's a bug
elsewhere, etc., but we should aim to handle this gracefully. This is
was the event timestamps are for. If, for example, you depend on getting
accurate timing data for touch events use these timestamps and not the
system time when you receive the event.

This is a big issue for me personally on OS X. I have a macbook with 2
GiB of ram, which isn't quite enough to have a couple apps and 15 tabs
open in chrome without causing some swapping. Unfortunately, the OS X
dock seems to use the current system time for timing instead of the
input event timestamps. This means that when I instantaneously click on
an app in the dock and my computer is swapping, the dock may not get the
up click for a few seconds and then it interprets it as a hold instead.
Bad Dock!

This is all to say that I don't think we can resolve all issues that
would cause a client to receive touch events long after the physical
events occurred. Thus, we should ensure that the protocol allows for
handling such cases gracefully, and I believe this protocol does.

Thanks for the feedback!

-- Chase