[RFC libinput] Add an API for touchpad gesture events

Fri Jan 23 03:18:01 PST 2015

Hi,

On 23-01-15 05:21, Peter Hutterer wrote:
> CC-ing Carlos too, I'd like to get his input here.
>
> On Thu, Jan 22, 2015 at 04:52:50PM +0100, Hans de Goede wrote:
>> For touchscreens we always send raw touch events to the compositor, and the
>> compositor or application toolkits do gesture recognition. This makes sense
>> because on a touchscreen which window / widget the touches are over is
>> important context to know to interpret gestures.
>>
>> On touchpads however we never send raw events since a touchpad is an absolute
>> device which primary function is to send pointer motion delta-s, so we always
>> need to do processing (and a lot of it) on the raw events.
>>
>> Moreover there is nothing underneath the finger which influences how to
>> interpret gestures, and there is a lot of touchpad and libinput configuration
>> specific context necessary for gesture recognition. E.g. is this a clickpad,
>> and if so are softbuttons or clickfinger used? What is the size of the
>> softbuttons? Is this a true multi-touch touchpad or a semi multi-touch touchpad
>> which only gives us a bounding box enclosing the fingers? Etc.
>>
>> So for touchpads it is better to do gesture processing in libinput, this commit
>> adds an initial implementation of a Gesture event API which only supports swipe
>> gestures, other gestures will be added later following the same model wrt,
>> having clear start and stop events and the number of fingers involved being
>> fixed once a gesture sequence starts.
>>
>> Signed-off-by: Hans de Goede <hdegoede at redhat.com>
>> ---
>>   src/libinput.h | 155 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++-
>>   1 file changed, 154 insertions(+), 1 deletion(-)
>>
>> diff --git a/src/libinput.h b/src/libinput.h
>> index 7b7a2db..1507772 100644
>> --- a/src/libinput.h
>> +++ b/src/libinput.h
>> @@ -173,7 +173,11 @@ enum libinput_event_type {
>>   	 * Signals the end of a set of touchpoints at one device sample
>>   	 * time. This event has no coordinate information attached.
>>   	 */
>> -	LIBINPUT_EVENT_TOUCH_FRAME
>> +	LIBINPUT_EVENT_TOUCH_FRAME,
>> +
>> +	LIBINPUT_EVENT_GESTURE_SWIPE_START = 600,
>> +	LIBINPUT_EVENT_GESTURE_SWIPE,
>
> To avoid accidental ambiguity, I'd use ...SWIPE_UPDATE here.
>
>> +	LIBINPUT_EVENT_GESTURE_SWIPE_END,
>>   };
>>
>>   /**
>> @@ -350,6 +354,19 @@ libinput_event_get_touch_event(struct libinput_event *event);
>>   /**
>>    * @ingroup event
>>    *
>> + * Return the gesture event that is this input event. If the event type does
>> + * not match the gesture event types, this function returns NULL.
>> + *
>> + * The inverse of this function is libinput_event_gesture_get_base_event().
>> + *
>> + * @return A gesture event, or NULL for other events
>> + */
>> +struct libinput_event_gesture *
>> +libinput_event_get_gesture_event(struct libinput_event *event);
>> +
>> +/**
>> + * @ingroup event
>> + *
>>    * Return the device event that is this input event. If the event type does
>>    * not match the device event types, this function returns NULL.
>>    *
>> @@ -884,6 +901,142 @@ struct libinput_event *
>>   libinput_event_touch_get_base_event(struct libinput_event_touch *event);
>>
>>   /**
>> + * @defgroup event_gesture Gesture events
>> + *
>> + * Gesture events are generated when a gesture is recognized on a touchpad.
>> + *
>> + * Gesture sequences always start with a LIBINPUT_EVENT_GESTURE_FOO_START
>> + * event. All following gesture events will be of the
>> + * LIBINPUT_EVENT_GESTURE_FOO type until a LIBINPUT_EVENT_GESTURE_FOO_STOP is
>> + * generated which signals the end of the gesture.
>> + */
>
> btw, we should add a @page for gestures at a later point but until we have
> the implementation we can skip it.
>
>> +
>> +/**
>> + * @ingroup event_gesture
>> + *
>> + * @return The event time for this event
>> + */
>> +uint32_t
>> +libinput_event_gesture_get_time(struct libinput_event_gesture *event);
>> +
>> +/**
>> + * @ingroup event_gesture
>> + *
>> + * @return The generic libinput_event of this event
>> + */
>> +struct libinput_event *
>> +libinput_event_gesture_get_base_event(struct libinput_event_gesture *event);
>> +
>> +/**
>> + * @ingroup event_gesture
>> + *
>> + * Return the number of fingers used for a gesture. This can be used e.g.
>> + * to differentiate between 3 or 4 finger swipes.
>> + *
>> + * This function can be called on all gesture events including
>> + * LIBINPUT_EVENT_GESTURE_FOO_START and the returned finger count value will
>> + * not change during a sequence. IOW libinput_event_gesture_get_finger_count
>> + * will only return a different value then a previous call after a
>> + * LIBINPUT_EVENT_GESTURE_FOO_STOP has been received.
>
> hehe. tbh, the IOW... sentence made it more confusing, I think you can skip
> it.

Ok, I will drop it.

>
>> + *
>> + * @return the number of fingers used for a gesture
>> + */
>> +enum libinput_key_state
>> +libinput_event_gesture_get_finger_count(struct libinput_event_gesture *event);
>> +
>> +/**
>> + * @ingroup event_gesture
>> + *
>> + * Return the delta between the last event and the current event. For gesture
>> + * events that are not of type @ref LIBINPUT_EVENT_GESTURE_SWIPE, this
>> + * function returns 0.
>> + *
>> + * If a device employs pointer acceleration, the delta returned by this
>> + * function is the accelerated delta.
>
> should we provide the unaccelerated data here as well? we do for pointer
> events.

Yes, good idea I will add that.
>
>> + *
>> + * Relative motion deltas are normalized to represent those of a device with
>> + * 1000dpi resolution. See @ref motion_normalization for more details.
>> + *
>> + * @note It is an application bug to call this function for events other than
>> + * @ref LIBINPUT_EVENT_GESTURE_SWIPE.
>> + *
>> + * @return the relative x movement since the last event
>> + */
>> +double
>> +libinput_event_gesture_get_dx(struct libinput_event_gesture *event);
>> +
>> +/**
>> + * @ingroup event_gesture
>> + *
>> + * Return the delta between the last event and the current event. For gesture
>> + * events that are not of type @ref LIBINPUT_EVENT_GESTURE_SWIPE, this
>> + * function returns 0.
>> + *
>> + * If a device employs pointer acceleration, the delta returned by this
>> + * function is the accelerated delta.
>> + *
>> + * Relative motion deltas are normalized to represent those of a device with
>> + * 1000dpi resolution. See @ref motion_normalization for more details.
>> + *
>> + * @note It is an application bug to call this function for events other than
>> + * @ref LIBINPUT_EVENT_GESTURE_SWIPE.
>> + *
>> + * @return the relative y movement since the last event
>> + */
>> +double
>> +libinput_event_gesture_get_dy(struct libinput_event_gesture *event);
>> +
>> +/**
>> + * @ingroup event_gesture
>> + *
>> + * Return the absolute x coordinate of the current center position of a
>> + * swipe gesture. This can be used e.g. to determine if a swipe is starting
>> + * close to a touchpad edge, or to synchronize an animation with how much
>
> s/much/many/ (I think)

Fixed.

>> + * percent of the width of a touchpad a swipe gesture has travelled.
>> + *
>> + * The returned value is in the range of 0.0 to 1.0 with 0.0 indicating that
>> + * the center is on the right edge of the touchpad, and 1.0 indicating that it
>> + * is on the left edge.
>> + *
>> + * For gesture events that are not of type
>> + * @ref LIBINPUT_EVENT_GESTURE_SWIPE_START, @ref LIBINPUT_EVENT_GESTURE_SWIPE
>> + * or @ref LIBINPUT_EVENT_GESTURE_SWIPE_STOP this function returns 0.
>> + *
>> + * @note It is an application bug to call this function for events other than
>> + * @ref LIBINPUT_EVENT_GESTURE_SWIPE_START, @ref LIBINPUT_EVENT_GESTURE_SWIPE
>> + * or @ref LIBINPUT_EVENT_GESTURE_SWIPE_STOP.
>> + *
>> + * @return the current absolute x coordinate
>> + */
>> +double
>> +libinput_event_gesture_get_absolute_x(struct libinput_event_gesture *event);
>
> this is a bit inconsistent with the other API where absolute_x returns a
> value in mm. I think we should do that here as well and provide
> get_absolute_x_transformed - callers that need it in relation to the
> touchpad size can just provide 1.0 as width/height.

I don't think mm is really interesting on a touchpad, and in many cases if we
say this is in mm we would be lying as we do not know the exact dimensions. I
see using (fake) mm-s here and then having users call get_absolute_x_transformed
with 100 or 1.0 to get the percentage they very likely want from the beginning
as just adding baggage without any gain.

The intend behind these absolute coordinates is for them to be used in e.g.
a virtual desktop switching animation when switching using a swipe, this way the
user can start a swipe, see the new virtual desktop, realize it is the wrong one
and abort, or if it is the right one continue, and then past say 2/3ths of the
touchpad the animation will show the new virtual desktop locking into place, and
the user will know he can stop swiping. Notice the key thing here is the swipe
reaching the 2/3th threshold, not some magical amount of (fake) mm-s.

Carlos, Jonas what do you think ?

>> +
>> +/**
>> + * @ingroup event_gesture
>> + *
>> + * Return the absolute y coordinate of the current center position of a
>> + * swipe gesture. This can be used e.g. to determine if a swipe is starting
>> + * close to a touchpad edge, or to synchronize an animation with how much
>
> s/much/many/

Fixed.

>
>> + * percent of the height of a touchpad a swipe gesture has travelled.
>> + *
>> + * The returned value is in the range of 0.0 to 1.0 with 0.0 indicating that
>> + * the center is on the top edge of the touchpad, and 1.0 indicating that it
>> + * is on the bottom edge.
>> + *
>> + * For gesture events that are not of type
>> + * @ref LIBINPUT_EVENT_GESTURE_SWIPE_START, @ref LIBINPUT_EVENT_GESTURE_SWIPE
>> + * or @ref LIBINPUT_EVENT_GESTURE_SWIPE_STOP this function returns 0.
>> + *
>> + * @note It is an application bug to call this function for events other than
>> + * @ref LIBINPUT_EVENT_GESTURE_SWIPE_START, @ref LIBINPUT_EVENT_GESTURE_SWIPE
>> + * or @ref LIBINPUT_EVENT_GESTURE_SWIPE_STOP.
>> + *
>> + * @return the current absolute y coordinate
>> + */
>> +double
>> +libinput_event_gesture_get_absolute_y(struct libinput_event_gesture *event);
>> +
>> +/**
>>    * @defgroup base Initialization and manipulation of libinput contexts
>>    */
>
> Just a comment, I think this should be good for other gestures as well.
> If we look at pinch/rotate to have a vector length and a center of gravity,
> then the dx/dy can represent the vector.
> so minus the nitpicks above this looks good.

Thanks.

>
> Cheers,
>     Peter
>

Regards,

Hans