[RFC libinput] Add an API for touchpad gesture events

[Carlos Garnacho]

Hey Peter,

On jue, 2015-01-29 at 21:54 +1000, Peter Hutterer wrote:
> On 29/01/2015 20:58 , Carlos Garnacho wrote:
> > On jue, 2015-01-29 at 16:26 +1000, Peter Hutterer wrote:
> >> On Wed, Jan 28, 2015 at 06:18:44PM +0000, Carlos Garnacho wrote:
> >>> Hey,
> >>>
> >>> On mié, 2015-01-28 at 15:51 +0100, Hans de Goede wrote:
> >>>> Hi,
> >>>>
> >>>> On 28-01-15 13:59, Carlos Garnacho wrote:
> >>>>> Hey Hans!,
> >>>>>
> >>>>> (sorry for chiming in late, things got busy around Fosdem...)
> >>>>>
> >>>>> On jue, 2015-01-22 at 16:52 +0100, Hans de Goede wrote:
> >>>>>> +	LIBINPUT_EVENT_GESTURE_SWIPE_START = 600,
> >>>>>> +	LIBINPUT_EVENT_GESTURE_SWIPE,
> >>>>>> +	LIBINPUT_EVENT_GESTURE_SWIPE_END,
> >>>>>>   };
> >>>>>>
> >>>>>
> >>>>> One thing I'm unsure about this approach. Is there any plan to support
> >>>>> continuation and chaining-up of gestures?
> >>>>>
> >>>>> Eg. what would happen if a 3fg gesture is started and the user rests a
> >>>>> 4th finger on the touchpad? would you get SWIPE_END for 3fg and
> >>>>> SWIPE_START for 4fg?
> >>>>
> >>>> If we detect that the finger is just resting, e.g. we think it is a palm,
> >>>> then the 3fg gesture continues. If we believe it is an active finger
> >>>> then yes you would get a SWIPE_END for 3fg and SWIPE_START for 4fg,
> >>>> note that the implementation will ignore short (in duration) changes,
> >>>> so e.g. a quick accidental brush with an extra finger, or shortly
> >>>> lifting one finger will be seen as the gesture continuing as normal.
> >>>>
> >>>>> How would the compositor know whether the gesture
> >>>>> finished in a non misleading way?
> >>>>
> >>>> By the LIBINPUT_EVENT_GESTURE_SWIPE_END, if we send that too soon, then
> >>>> that would be a libinput implementation bug.
> >>>>
> >>>>> What I would expect it to happen is that the compositor cancelled the
> >>>>> reaction to the 3fg swipe in this situation.
> >>>>
> >>>> Right.
> >>>
> >>> My concern is, where is this expected to happen? If you get SWIPE_END
> >>> for 3fg before SWIPE_BEGIN for 4fg, how can you tell whether the gesture
> >>> actually finished or shifted into something else, other than waiting if
> >>> something else happens after?
> >>
> >> honestly, I'm not a big fan of this approach. I'd prefer it if a gesture was
> >> locked to one gesture for its lifetime, regardless of whether we're adding
> >> or removing fingers. obviously dropping to 1 finger should cancel the
> >> ongoing gesture.
> > 
> > I would be fine with such gesture locking, as long as cancellation is
> > properly communicated.
> > 
> >>
> >> I'm struggling to find use-cases where moving from one gesture to another
> >> really makes sense and is something that is obvious to the user too. And it
> >> introduces complexity in the API that we don't really want, IMO.
> >>
> >> What are the use-cases for gesture transitions? (and note we're talking
> >> about a touchpad here, not a touchscreen where the conditions are a bit
> >> different).
> > 
> > Those have indeed some differences. My usecase here is mainly "the user
> > didn't mean to", and I wouldn't like it if compositors were unable to
> > behave in a forgiving/non-disrupting way.
> > 
> > It is true that most triggered operations don't fall in this category,
> > and are safe to cancel when a gesture starts (eg. cancelling ongoing
> > kinetic scrolling when a pinch/zoom gesture starts), but compositor
> > gestures will usually be one-off, context-switching operations, I'd find
> > it rather harsh if there were no way out of that.
> 
> Isn't this mostly a UI issue than the gesture itself? Most gestures you
> "undo" by doing the reverse. when you start pinching, expanding the
> fingers restores you back to the original state. A slow swipe would be
> the same, a fast swipe is likely over before you realise it's the wrong
> thing to do :)
> 
> So in both cases it's the compositor/client that needs to realize that
> the gesture was undone and behave accordingly. Gestures that are
> "binary" (e.g. start something, show the overview, etc.) will need some
> sort of trigger threshold at which they kick in (and you can ease that
> by hinting something will happen before the threshold kicks in).
> 
> Either way, all that heavily depends on semantics that libinput doesn't
> have access to. So I'm not sure at the moment what you envision as
> gesture cancellation.

Ok :), let's try calling it "gesture shifting", I'll try to picture it
with an specific example: I sloppily start a 4 finger swipe, in a way
that 3 fingers start within the touchpad area and the 4th lies slightly
outside. As I swipe the 4th finger enters the touchpad area and is thus
detected too.

Now let's say libinput already started recognizing the 3fg gesture, at
this point, either triggering SWIPE_END for 3fg immediately and
switching to 4fg, or locking on the first 3fg gesture and ignoring the
4th finger would result in an action I didn't consciously make.

Of course you can punt this to the compositor, and trust they have big
enough thresholds to avoid falling into this mistake once the 3fg
gesture "ends", I think such "hmm, the user probably didn't mean this"
should be made explicit though.

The challenges I see with touchpads is that they have a rather limited
physical area, they're something you don't usually look at directly when
operating, user dexterity varies, and not all have bevels to help your
muscle memory. IMO this all makes such situations not too unlikely.

Cheers,
  Carlos