[RFC libinput] Add an API for touchpad gesture events

[Peter Hutterer]

On Thu, Jan 29, 2015 at 06:42:54PM +0100, Hans de Goede wrote:
> Hi,
> 
> On 29-01-15 16:23, Carlos Garnacho wrote:
> >Hey Peter,
> >
> >On jue, 2015-01-29 at 21:54 +1000, Peter Hutterer wrote:
> 
> <snip>
> 
> >>Isn't this mostly a UI issue than the gesture itself? Most gestures you
> >>"undo" by doing the reverse. when you start pinching, expanding the
> >>fingers restores you back to the original state. A slow swipe would be
> >>the same, a fast swipe is likely over before you realise it's the wrong
> >>thing to do :)
> >>
> >>So in both cases it's the compositor/client that needs to realize that
> >>the gesture was undone and behave accordingly. Gestures that are
> >>"binary" (e.g. start something, show the overview, etc.) will need some
> >>sort of trigger threshold at which they kick in (and you can ease that
> >>by hinting something will happen before the threshold kicks in).
> >>
> >>Either way, all that heavily depends on semantics that libinput doesn't
> >>have access to. So I'm not sure at the moment what you envision as
> >>gesture cancellation.
> >
> >Ok :), let's try calling it "gesture shifting", I'll try to picture it
> >with an specific example: I sloppily start a 4 finger swipe, in a way
> >that 3 fingers start within the touchpad area and the 4th lies slightly
> >outside. As I swipe the 4th finger enters the touchpad area and is thus
> >detected too.
> >
> >Now let's say libinput already started recognizing the 3fg gesture, at
> >this point, either triggering SWIPE_END for 3fg immediately and
> >switching to 4fg, or locking on the first 3fg gesture and ignoring the
> >4th finger would result in an action I didn't consciously make.
> >
> >Of course you can punt this to the compositor, and trust they have big
> >enough thresholds to avoid falling into this mistake once the 3fg
> >gesture "ends", I think such "hmm, the user probably didn't mean this"
> >should be made explicit though.
> >
> >The challenges I see with touchpads is that they have a rather limited
> >physical area, they're something you don't usually look at directly when
> >operating, user dexterity varies, and not all have bevels to help your
> >muscle memory. IMO this all makes such situations not too unlikely.
> 
> The way I see this is really simple:
> 
> 1) the user has made a mistake / wrongly operated the computer
> 2) the computer as a result does not do what the computer wants
> 
> This is basically expected behavior, if you mistype, you get the wrong
> letter, if with a mouse you accidentally click next to where you intended
> to click, the computer / software behaves as if you deliberately misclicked.
> 
> What you're asking for is a crystal ball / a DWIM (do what I mean) computer,
> and those simply do not exist. If we at all sort of magic heuristics to try
> and correct user errors like the above, all that we will end up is so much
> magic that the user will no longer understand why the computer is responding
> the way it is.
> 
> Where as if we do not try to be smart the user will think, ok one of my fingers
> was not actually on the touchpad, I better pay attention that I do not do that
> again.
> 
> This only becomes a problem is the result of the unintended gesture is not
> undo-able, and frankly if a UI has things triggered by gestures which cannot
> be (easily) undone, then that is simply a BAD UI.

I agree with Hans there, this seems to be something we should punt to the
user. There is some room for crystal-balls within libinput [1] but overall I
think this is going too far.

Again, the difference between touchpads and touchscreens come into play
here. On touchscreens you get no haptic feedback when you miss a window
border. On all touchpads that I have used so far you get some haptic
feedback for whether you're on the touchpad or not, or at least you get some
haptic feedback when you cross the touchpad bevel - however small that may
be. That should trigger the user's realization that something didn't quite
work out.

This is in response to your specific example. I think the big takeaway
though is: gestures are unreliable. And the UI needs to be designed with
this in mind, i.e. use only gestures that have little room for error.

A four-finger gesture is not precise for the reasons you mentioned above.
The solution to that is to design the UI that this gesture cannot fail.
So for example, if you want a 4-finger swipe, make it clear to the user that
the gesture wanted should start in the middle of the touchpad and then
swipes left/right from there, not across the whole touchpad.
And of course have rollback options for any gesture.

To use Apple as example again: there you only get to chose from a limited
set of gestures for each feature and the gestures are balanced so that they
don't easily interfere.

Cheers,
   Peter

[1] depending on the location and speed of the finger movement, we _may_ be
able to delay the gesture until we're sure that 4th finger doesn't come in.