[RFC libinput] Add an API for touchpad gesture events

[Peter Hutterer]

On Wed, Jan 28, 2015 at 04:02:20PM +0100, Hans de Goede wrote:
> Hi,
> 
> On 28-01-15 14:02, Carlos Garnacho wrote:
> >Hey Hans,
> >
> >On mié, 2015-01-28 at 08:38 +0100, Hans de Goede wrote:
> 
> <snip>
> 
> >>>Ok, I'll change this to mm for v3 then (just send v2 addressing the other
> >>>review comments and adding a core implementation).
> >>
> >>So I've been thinking more about this and I'm still not fully convinced, let
> >>me explain.
> >>
> >>The problem is that the edges where a swipe begins and end may vary based
> >>on config settings and even per swipe.
> >>
> >>e.g. If a clickpad is using softbutton areas on the bottom, and the user
> >>swipes up, the swipe gesture will not start until the fingers have left the
> >>bottom button area, because before that they are seen as fingers resting to
> >>click the buttons, so for that gesture the bottom edge would be the top edge
> >>of the button area.
> >>
> >>But if the user swipes down, he can use the bottom button area because fingers
> >>are only not counted as active there, when they've started there.
> >>
> >>Now we could always only count the non soft-button area as the area to which
> >>absolute gesture coordinates should be compared to see where on the touchpad
> >>the gesture is, but what if clickfinger is enabled, then there is no soft
> >>button area?  Then what if the user changes the clickfinger setting ?
> >>
> >>A solution to this would be putting the relevant area in the gesture event,
> >>and documenting that it may differ per event and that users should always
> >>compare absolute coordinates to the area relevant for that device. But this
> >>does not sound like a very nice API to me, and also makes me worry about the
> >>wayland protocol.
> >
> >I'm maybe missing something, but how is the area relevant for anything
> >else than finding out whether a touch is eligible to start a gesture?
> >After that all that should matter is the relative position of the
> >involved touches.
> 
> Quoting from the actual patch you're replying to:
> 
> "* Return the absolute x coordinate of the current center position of a
>  * swipe gesture. This can be used e.g. to determine if a swipe is starting
>  * close to a touchpad edge, or to synchronize an animation with how many
>  * percent of the width of a touchpad a swipe gesture has traveled."
> 
> AFAIK mac os x does something like this (animation based on how far along
> the touchpad a swipe gesture has moved) for some transations, I want to
> keep this use case open in the API.

to clarify those two, from what I can tell on OS X 10.9.5:
- there is a "swipe left from the right edge with two fingers" gesture that
can be selected to show the Notification Center. That's the only one that
has some spatial arrangement in it.
- the animations aren't tied to the width of the touchpad, the one where the
animation follows the finger merely uses 2-3 cm of the touchpad while
pinching. afaict nothing uses the actual width of the touchpad directly.

sorry if these weren't clear in the original emails where we discussed this
note that at least in the system preferences the selection of gestures
available is relatively small and contained, it's not a free-for-all.

There is one gesture for three-finger dragging but that seems to be the
equivalent of button-down, pointer motion rather than being a direct mapping
of the touchpad to the screen area or somesuch.

> >>All in all this sounds very much NOT kiss, and I'm a big fan of kiss, so I
> >>really believe that the initially proposed 0.0 - 1.0 scale is much better,
> >
> >But we get back here to the stretched rectangle case :(. IMO there
> >should be a way to know about the proper w/h ratios at least.
> 
> This is highly unlikely to be used in a mode where we're not only interested
> in either vertical or horizontal movement.
> 
> Note this is only about swipes, later on a separate gesture event type will
> be added for zoom/rotate, and there only the deltas matter, and those use
> a normalized resolution, so that a square gesture on the touchpad results
> in identical dx and dy events / scales.
> 
> >>it already gives apps way more info then the current gtk and mac os x api-s
> >>do, as those both only give deltas, while turning all the above into an
> >>implentation detail, rather then something which we need to explain in API
> >>docs. The above is exactly why we want to do the gesture handling inside
> >>libinput in the first place, and exporting absolute event data in mm, unhides
> >>all the nitty gritty we're trying to abstract away in the first place.
> >
> >I don't think it's incompatible, given that libinput determines first
> >whether the touch is part of a gesture before letting know of it.
> 
> Lets say the compositor will see an upward vertical swipe as being relevant
> if it starts below a line 10% of the touchpad height from the bottom, but we
> never send events for that because we do not start seeing the fingers as
> active until the leave the bottom soft button area which is 12% high...
> 
> As said doing this in mm means that we need to send the relevant bounding
> box within which the gesture sequence can send coordinates per gesture
> as it can change based on direction of the gesture (upward vs downward)
> and on config settings (clickfinger vs softbuttons), this is possible, but
> does not seem like a nice API to me, also because it is counter intuitive
> and users may very well get the bounding box once and then store it, as
> they will expect it to stay the same.
> 
> The only real argument against the 0.0 - 1.0 scale which I've heard otoh
> is that it looses aspect ratio info, which tends to be utterly irrelevant
> for swipe gestures anyways, and if it is relevant then the application
> can use the delta events, which are aspect ratio correct.

I'm beginning to think that maybe having a set of tags may be better here.
e.g. left edge, right edge, corner, etc.
This absolves us from any requirements for precision but still allows some
use-cases. The only one that won't work is to make things dependent on the
touchpad width.

Adding an edge flag is easy to do retroactively too, so if the gesture
starts at the bottom and we don't recognise it as gesture until it goes
outside of the button area - that still allows us to apply the edge tag
without messing around with coordinate scales.

Cheers,
   Peter