[RFC libinput] Add an API for touchpad gesture events

Fri Jan 23 05:53:58 PST 2015

On Fri, Jan 23, 2015 at 12:18:01PM +0100, Hans de Goede wrote:
> Hi,
> 
> On 23-01-15 05:21, Peter Hutterer wrote:
> >CC-ing Carlos too, I'd like to get his input here.
> >
> >On Thu, Jan 22, 2015 at 04:52:50PM +0100, Hans de Goede wrote:
> >>For touchscreens we always send raw touch events to the compositor, and the
> >>compositor or application toolkits do gesture recognition. This makes sense
> >>because on a touchscreen which window / widget the touches are over is
> >>important context to know to interpret gestures.
> >>
> >>On touchpads however we never send raw events since a touchpad is an absolute
> >>device which primary function is to send pointer motion delta-s, so we always
> >>need to do processing (and a lot of it) on the raw events.
> >>
> >>Moreover there is nothing underneath the finger which influences how to
> >>interpret gestures, and there is a lot of touchpad and libinput configuration
> >>specific context necessary for gesture recognition. E.g. is this a clickpad,
> >>and if so are softbuttons or clickfinger used? What is the size of the
> >>softbuttons? Is this a true multi-touch touchpad or a semi multi-touch touchpad
> >>which only gives us a bounding box enclosing the fingers? Etc.
> >>
> >>So for touchpads it is better to do gesture processing in libinput, this commit
> >>adds an initial implementation of a Gesture event API which only supports swipe
> >>gestures, other gestures will be added later following the same model wrt,
> >>having clear start and stop events and the number of fingers involved being
> >>fixed once a gesture sequence starts.
> >>
> >>Signed-off-by: Hans de Goede <hdegoede at redhat.com>
> >>---
> >>  src/libinput.h | 155 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++-
> >>  1 file changed, 154 insertions(+), 1 deletion(-)
> >>
> >>diff --git a/src/libinput.h b/src/libinput.h
> >>index 7b7a2db..1507772 100644
> >>--- a/src/libinput.h
> >>+++ b/src/libinput.h
> >>@@ -173,7 +173,11 @@ enum libinput_event_type {
> >>  	 * Signals the end of a set of touchpoints at one device sample
> >>  	 * time. This event has no coordinate information attached.
> >>  	 */
> >>-	LIBINPUT_EVENT_TOUCH_FRAME
> >>+	LIBINPUT_EVENT_TOUCH_FRAME,
> >>+
> >>+	LIBINPUT_EVENT_GESTURE_SWIPE_START = 600,
> >>+	LIBINPUT_EVENT_GESTURE_SWIPE,
> >
> >To avoid accidental ambiguity, I'd use ...SWIPE_UPDATE here.
> >
> >>+	LIBINPUT_EVENT_GESTURE_SWIPE_END,
> >>  };
> >>
> >>  /**
> >>@@ -350,6 +354,19 @@ libinput_event_get_touch_event(struct libinput_event *event);
> >>  /**
> >>   * @ingroup event
> >>   *
> >>+ * Return the gesture event that is this input event. If the event type does
> >>+ * not match the gesture event types, this function returns NULL.
> >>+ *
> >>+ * The inverse of this function is libinput_event_gesture_get_base_event().
> >>+ *
> >>+ * @return A gesture event, or NULL for other events
> >>+ */
> >>+struct libinput_event_gesture *
> >>+libinput_event_get_gesture_event(struct libinput_event *event);
> >>+
> >>+/**
> >>+ * @ingroup event
> >>+ *
> >>   * Return the device event that is this input event. If the event type does
> >>   * not match the device event types, this function returns NULL.
> >>   *
> >>@@ -884,6 +901,142 @@ struct libinput_event *
> >>  libinput_event_touch_get_base_event(struct libinput_event_touch *event);
> >>
> >>  /**
> >>+ * @defgroup event_gesture Gesture events
> >>+ *
> >>+ * Gesture events are generated when a gesture is recognized on a touchpad.
> >>+ *
> >>+ * Gesture sequences always start with a LIBINPUT_EVENT_GESTURE_FOO_START
> >>+ * event. All following gesture events will be of the
> >>+ * LIBINPUT_EVENT_GESTURE_FOO type until a LIBINPUT_EVENT_GESTURE_FOO_STOP is
> >>+ * generated which signals the end of the gesture.
> >>+ */
> >
> >btw, we should add a @page for gestures at a later point but until we have
> >the implementation we can skip it.
> >
> >>+
> >>+/**
> >>+ * @ingroup event_gesture
> >>+ *
> >>+ * @return The event time for this event
> >>+ */
> >>+uint32_t
> >>+libinput_event_gesture_get_time(struct libinput_event_gesture *event);
> >>+
> >>+/**
> >>+ * @ingroup event_gesture
> >>+ *
> >>+ * @return The generic libinput_event of this event
> >>+ */
> >>+struct libinput_event *
> >>+libinput_event_gesture_get_base_event(struct libinput_event_gesture *event);
> >>+
> >>+/**
> >>+ * @ingroup event_gesture
> >>+ *
> >>+ * Return the number of fingers used for a gesture. This can be used e.g.
> >>+ * to differentiate between 3 or 4 finger swipes.
> >>+ *
> >>+ * This function can be called on all gesture events including
> >>+ * LIBINPUT_EVENT_GESTURE_FOO_START and the returned finger count value will
> >>+ * not change during a sequence. IOW libinput_event_gesture_get_finger_count
> >>+ * will only return a different value then a previous call after a
> >>+ * LIBINPUT_EVENT_GESTURE_FOO_STOP has been received.
> >
> >hehe. tbh, the IOW... sentence made it more confusing, I think you can skip
> >it.
> 
> Ok, I will drop it.
> 
> >
> >>+ *
> >>+ * @return the number of fingers used for a gesture
> >>+ */
> >>+enum libinput_key_state
> >>+libinput_event_gesture_get_finger_count(struct libinput_event_gesture *event);
> >>+
> >>+/**
> >>+ * @ingroup event_gesture
> >>+ *
> >>+ * Return the delta between the last event and the current event. For gesture
> >>+ * events that are not of type @ref LIBINPUT_EVENT_GESTURE_SWIPE, this
> >>+ * function returns 0.
> >>+ *
> >>+ * If a device employs pointer acceleration, the delta returned by this
> >>+ * function is the accelerated delta.
> >
> >should we provide the unaccelerated data here as well? we do for pointer
> >events.
> 
> Yes, good idea I will add that.
> >
> >>+ *
> >>+ * Relative motion deltas are normalized to represent those of a device with
> >>+ * 1000dpi resolution. See @ref motion_normalization for more details.
> >>+ *
> >>+ * @note It is an application bug to call this function for events other than
> >>+ * @ref LIBINPUT_EVENT_GESTURE_SWIPE.
> >>+ *
> >>+ * @return the relative x movement since the last event
> >>+ */
> >>+double
> >>+libinput_event_gesture_get_dx(struct libinput_event_gesture *event);
> >>+
> >>+/**
> >>+ * @ingroup event_gesture
> >>+ *
> >>+ * Return the delta between the last event and the current event. For gesture
> >>+ * events that are not of type @ref LIBINPUT_EVENT_GESTURE_SWIPE, this
> >>+ * function returns 0.
> >>+ *
> >>+ * If a device employs pointer acceleration, the delta returned by this
> >>+ * function is the accelerated delta.
> >>+ *
> >>+ * Relative motion deltas are normalized to represent those of a device with
> >>+ * 1000dpi resolution. See @ref motion_normalization for more details.
> >>+ *
> >>+ * @note It is an application bug to call this function for events other than
> >>+ * @ref LIBINPUT_EVENT_GESTURE_SWIPE.
> >>+ *
> >>+ * @return the relative y movement since the last event
> >>+ */
> >>+double
> >>+libinput_event_gesture_get_dy(struct libinput_event_gesture *event);
> >>+
> >>+/**
> >>+ * @ingroup event_gesture
> >>+ *
> >>+ * Return the absolute x coordinate of the current center position of a
> >>+ * swipe gesture. This can be used e.g. to determine if a swipe is starting
> >>+ * close to a touchpad edge, or to synchronize an animation with how much
> >
> >s/much/many/ (I think)
> 
> Fixed.
> 
> >>+ * percent of the width of a touchpad a swipe gesture has travelled.
> >>+ *
> >>+ * The returned value is in the range of 0.0 to 1.0 with 0.0 indicating that
> >>+ * the center is on the right edge of the touchpad, and 1.0 indicating that it
> >>+ * is on the left edge.
> >>+ *
> >>+ * For gesture events that are not of type
> >>+ * @ref LIBINPUT_EVENT_GESTURE_SWIPE_START, @ref LIBINPUT_EVENT_GESTURE_SWIPE
> >>+ * or @ref LIBINPUT_EVENT_GESTURE_SWIPE_STOP this function returns 0.
> >>+ *
> >>+ * @note It is an application bug to call this function for events other than
> >>+ * @ref LIBINPUT_EVENT_GESTURE_SWIPE_START, @ref LIBINPUT_EVENT_GESTURE_SWIPE
> >>+ * or @ref LIBINPUT_EVENT_GESTURE_SWIPE_STOP.
> >>+ *
> >>+ * @return the current absolute x coordinate
> >>+ */
> >>+double
> >>+libinput_event_gesture_get_absolute_x(struct libinput_event_gesture *event);
> >
> >this is a bit inconsistent with the other API where absolute_x returns a
> >value in mm. I think we should do that here as well and provide
> >get_absolute_x_transformed - callers that need it in relation to the
> >touchpad size can just provide 1.0 as width/height.
> 
> I don't think mm is really interesting on a touchpad, and in many cases if we
> say this is in mm we would be lying as we do not know the exact dimensions. I
> see using (fake) mm-s here and then having users call get_absolute_x_transformed
> with 100 or 1.0 to get the percentage they very likely want from the beginning
> as just adding baggage without any gain.
> 
> The intend behind these absolute coordinates is for them to be used in e.g.
> a virtual desktop switching animation when switching using a swipe, this way the
> user can start a swipe, see the new virtual desktop, realize it is the wrong one
> and abort, or if it is the right one continue, and then past say 2/3ths of the
> touchpad the animation will show the new virtual desktop locking into place, and
> the user will know he can stop swiping. Notice the key thing here is the swipe
> reaching the 2/3th threshold, not some magical amount of (fake) mm-s.
> 
> Carlos, Jonas what do you think ?

That would still be possible with a mm based absolute coordinate as we
can retrieve the physical dimension of the touchpad with
libinput_device_get_size(). And as long as the driver gives us reasonably
correct physical dimension data, the mm-s aren't fake; or do you mean
most drivers lie to us? It'd be better also not to have it simply 1.0 x
1.0, as that'd make making a circle on the device result in an ellipse in
the API.

Since we already do mm-s everywhere, I don't see it that horrible to not
drop that information here in order for some smart future usage, or at
least consistency.

Jonas

> 
> >>+
> >>+/**
> >>+ * @ingroup event_gesture
> >>+ *
> >>+ * Return the absolute y coordinate of the current center position of a
> >>+ * swipe gesture. This can be used e.g. to determine if a swipe is starting
> >>+ * close to a touchpad edge, or to synchronize an animation with how much
> >
> >s/much/many/
> 
> Fixed.
> 
> >
> >>+ * percent of the height of a touchpad a swipe gesture has travelled.
> >>+ *
> >>+ * The returned value is in the range of 0.0 to 1.0 with 0.0 indicating that
> >>+ * the center is on the top edge of the touchpad, and 1.0 indicating that it
> >>+ * is on the bottom edge.
> >>+ *
> >>+ * For gesture events that are not of type
> >>+ * @ref LIBINPUT_EVENT_GESTURE_SWIPE_START, @ref LIBINPUT_EVENT_GESTURE_SWIPE
> >>+ * or @ref LIBINPUT_EVENT_GESTURE_SWIPE_STOP this function returns 0.
> >>+ *
> >>+ * @note It is an application bug to call this function for events other than
> >>+ * @ref LIBINPUT_EVENT_GESTURE_SWIPE_START, @ref LIBINPUT_EVENT_GESTURE_SWIPE
> >>+ * or @ref LIBINPUT_EVENT_GESTURE_SWIPE_STOP.
> >>+ *
> >>+ * @return the current absolute y coordinate
> >>+ */
> >>+double
> >>+libinput_event_gesture_get_absolute_y(struct libinput_event_gesture *event);
> >>+
> >>+/**
> >>   * @defgroup base Initialization and manipulation of libinput contexts
> >>   */
> >
> >Just a comment, I think this should be good for other gestures as well.
> >If we look at pinch/rotate to have a vector length and a center of gravity,
> >then the dx/dy can represent the vector.
> >so minus the nitpicks above this looks good.
> 
> 
> Thanks.
> 
> 
> >
> >Cheers,
> >    Peter
> >
> 
> Regards,
> 
> Hans
> _______________________________________________
> wayland-devel mailing list
> wayland-devel at lists.freedesktop.org
> http://lists.freedesktop.org/mailman/listinfo/wayland-devel