Multitouch protocol and misbehaving clients

[Peter Hutterer]

On Wed, Sep 07, 2011 at 10:01:06AM -0700, Chase Douglas wrote:
> On 08/31/2011 11:06 AM, Carlos Garnacho wrote:
> > Hello,
> > 
> > I've been following the latest changes to the XInput protocol for
> > multitouch and doing practical experiments with the Ubuntu shipped XI2.1
> > on Mutter/Gnome-shell.
> 
> Great!
> 
> > Overall, the protocol description looks quite polished now, although I'm
> > quite concerned about the strict dependence of the server on grabbing
> > clients calling XIAllowTouchEvents(), and how it has no way to handle
> > misbehaving clients.
> > 
> > Here are a few points I think it'd be good to restrain and clarify:
> > 
> >       * What happens if a client fails to accept/reject a touch
> >         sequence?
> 
> Nothing happens until the client either: fails to accept/reject a touch
> or disappears (in which case the touches are rejected).

correct. I'd also like to point out that this is the same behaviour we
already have for pointer grabs where it's easy to get a client to hold the
pointer hostage.

in that context: we do have a field specifying the number of touches a
device supports. we should amend the documentation stating that in the case
of TouchPendingEnd, more than that number of touches can be active since
that number refers to physical, not logical touches.

don't really have anything to add to the rest. Misbehaving clients can hold
up the X server and that's always been the case. I think the protocol has
room for dealing with this without killing the server.

are there any examples you noticed that weren't caused by actual bugs but
could be design issues?

Cheers,
  Peter

> >       * What happens if a missed, physically finished touch sequence
> >         owned the pointer emulation? should pointer emulation be carried
> >         over to a newly started touchpoint? The XI2proto changes mention
> >         that pointer emulation is "implementation defined", so this
> >         might fall into such implementation details.
> 
> The prototype server in Ubuntu would essentially hold the pointer hostage.
> 
> > Perhaps these two issues should just be dealt with as stale active
> > grabs, and release the touch sequences as soon as the client that's
> > stuck with these disappears. 
> > 
> > However, the unbounded amount of stale resources the server could create
> > for unfinished touch sequences until the client quits is somewhat
> > concerning.
> 
> Just to clarify here, the server may bound the amount of resources given
> for touch handling. For example, in the prototype server in Ubuntu there
> is a ring buffer of touch events for each touch sequence. If you
> overflow the ring buffer, older touch update events are discarded (the
> begin event is kept so future clients will know where the touch started
> at least). This is detailed in the spec, that touch update events may be
> dropped.
> 
> > And in that situation, just transferring ownership onto the
> > next client not listening to unowned events might mean lots of old, out
> > of context touches that would be misinterpreted as "recent enough" when
> > these are replayed.
> 
> This can be an issue no matter what you do. However, we have the same
> issue through other means too. If you have a machine that is really
> loaded and swapping constantly, the desktop may hang for a bit. We could
> throw our hands up and say the user should have more ram, or it's a bug
> elsewhere, etc., but we should aim to handle this gracefully. This is
> was the event timestamps are for. If, for example, you depend on getting
> accurate timing data for touch events use these timestamps and not the
> system time when you receive the event.
> 
> This is a big issue for me personally on OS X. I have a macbook with 2
> GiB of ram, which isn't quite enough to have a couple apps and 15 tabs
> open in chrome without causing some swapping. Unfortunately, the OS X
> dock seems to use the current system time for timing instead of the
> input event timestamps. This means that when I instantaneously click on
> an app in the dock and my computer is swapping, the dock may not get the
> up click for a few seconds and then it interprets it as a hold instead.
> Bad Dock!

> This is all to say that I don't think we can resolve all issues that
> would cause a client to receive touch events long after the physical
> events occurred. Thus, we should ensure that the protocol allows for
> handling such cases gracefully, and I believe this protocol does.
> 
> Thanks for the feedback!
> 
> -- Chase
> _______________________________________________
> xorg-devel at lists.x.org: X.Org development
> Archives: http://lists.x.org/archives/xorg-devel
> Info: http://lists.x.org/mailman/listinfo/xorg-devel
>