[RFC libinput v2] Add an API for touchpad gesture events
Hans de Goede
hdegoede at redhat.com
Sat Jan 24 00:41:56 PST 2015
Hi,
On 23-01-15 16:17, Derek Foreman wrote:
> On 23/01/15 07:46 AM, Hans de Goede wrote:
>> For touchscreens we always send raw touch events to the compositor, and the
>> compositor or application toolkits do gesture recognition. This makes sense
>> because on a touchscreen which window / widget the touches are over is
>> important context to know to interpret gestures.
>>
>> On touchpads however we never send raw events since a touchpad is an absolute
>> device which primary function is to send pointer motion delta-s, so we always
>> need to do processing (and a lot of it) on the raw events.
>>
>> Moreover there is nothing underneath the finger which influences how to
>> interpret gestures, and there is a lot of touchpad and libinput configuration
>> specific context necessary for gesture recognition. E.g. is this a clickpad,
>> and if so are softbuttons or clickfinger used? What is the size of the
>> softbuttons? Is this a true multi-touch touchpad or a semi multi-touch touchpad
>> which only gives us a bounding box enclosing the fingers? Etc.
>>
>> So for touchpads it is better to do gesture processing in libinput, this commit
>> adds an initial implementation of a Gesture event API which only supports swipe
>> gestures, other gestures will be added later following the same model wrt,
>> having clear start and stop events and the number of fingers involved being
>> fixed once a gesture sequence starts.
>
> It seems to me that different desktop environments/compositors/whatever
> might not all want the same gestures, and if someone comes up with some
> "clever" new gesture, they might not want to fight with libinput patch
> review in order to bring it to their users.
>
> I'm also wondering how "standard" gestures will be interpreted. What if
> I want to be able to pinch/twist to zoom and rotate with a single
> gesture?
We will provide both finger distance and angle in a single event, so yes
that will be possible.
> What if I don't? Will libinput be flexible enough to
> accommodate both cases?
If you do not then you will need to chose a mode based on the initial
event(s) and then ignore the other data for the duration of that
sequence, so yet that will be possible too.
We're doing this this way deliberately, because zoom/rotate gestures
are typically not used by the compositor, but send to the focussed
app, and the desired mode may differ per app.
>
> Will there be some way for a compositor to turn off this gesture
> processing and do its own (presumably on raw touchpad data)?
Yes we plan a full raw mode, but in this case we go 100% raw, touchpads
are complex beasts to work with actually, which is why we believe it
is best to do the processing in libinput, but if you want raw in
the future we will provide raw, for e.g. apps like handwriting on the
touchpad. As said we will go 100% raw then, and if you still somehow
want the touchpad to also generate pointer movement events you will
need to do that yourself.
>
> (a couple of small comments below as well)
>
>> Signed-off-by: Hans de Goede <hdegoede at redhat.com>
>> ---
>> src/libinput-private.h | 12 +++
>> src/libinput.c | 147 ++++++++++++++++++++++++++++++++++++
>> src/libinput.h | 199 ++++++++++++++++++++++++++++++++++++++++++++++++-
>> 3 files changed, 357 insertions(+), 1 deletion(-)
>>
>> diff --git a/src/libinput-private.h b/src/libinput-private.h
>> index b938ed0..0243aa6 100644
>> --- a/src/libinput-private.h
>> +++ b/src/libinput-private.h
>> @@ -317,6 +317,18 @@ touch_notify_touch_up(struct libinput_device *device,
>> int32_t seat_slot);
>>
>> void
>> +gesture_notify_swipe(struct libinput_device *device,
>> + uint64_t time,
>> + enum libinput_event_type type,
>> + int finger_count,
>> + double dx,
>> + double dy,
>> + double dx_unaccel,
>> + double dy_unaccel,
>> + double abs_x,
>> + double abs_y);
>> +
>> +void
>> touch_notify_frame(struct libinput_device *device,
>> uint64_t time);
>>
>> diff --git a/src/libinput.c b/src/libinput.c
>> index 951698a..51b316a 100644
>> --- a/src/libinput.c
>> +++ b/src/libinput.c
>> @@ -79,6 +79,18 @@ struct libinput_event_touch {
>> double y;
>> };
>>
>> +struct libinput_event_gesture {
>> + struct libinput_event base;
>> + uint32_t time;
>> + int finger_count;
>> + double dx;
>> + double dy;
>> + double dx_unaccel;
>> + double dy_unaccel;
>> + double abs_x;
>> + double abs_y;
>> +};
>> +
>> static void
>> libinput_default_log_func(struct libinput *libinput,
>> enum libinput_log_priority priority,
>> @@ -183,6 +195,10 @@ libinput_event_get_pointer_event(struct libinput_event *event)
>> case LIBINPUT_EVENT_TOUCH_CANCEL:
>> case LIBINPUT_EVENT_TOUCH_FRAME:
>> break;
>> + case LIBINPUT_EVENT_GESTURE_SWIPE_START:
>> + case LIBINPUT_EVENT_GESTURE_SWIPE_UPDATE:
>> + case LIBINPUT_EVENT_GESTURE_SWIPE_END:
>> + break;
>> }
>>
>> return NULL;
>> @@ -209,6 +225,10 @@ libinput_event_get_keyboard_event(struct libinput_event *event)
>> case LIBINPUT_EVENT_TOUCH_CANCEL:
>> case LIBINPUT_EVENT_TOUCH_FRAME:
>> break;
>> + case LIBINPUT_EVENT_GESTURE_SWIPE_START:
>> + case LIBINPUT_EVENT_GESTURE_SWIPE_UPDATE:
>> + case LIBINPUT_EVENT_GESTURE_SWIPE_END:
>> + break;
>> }
>>
>> return NULL;
>> @@ -234,6 +254,40 @@ libinput_event_get_touch_event(struct libinput_event *event)
>> case LIBINPUT_EVENT_TOUCH_CANCEL:
>> case LIBINPUT_EVENT_TOUCH_FRAME:
>> return (struct libinput_event_touch *) event;
>> + case LIBINPUT_EVENT_GESTURE_SWIPE_START:
>> + case LIBINPUT_EVENT_GESTURE_SWIPE_UPDATE:
>> + case LIBINPUT_EVENT_GESTURE_SWIPE_END:
>> + break;
>> + }
>> +
>> + return NULL;
>> +}
>> +
>> +LIBINPUT_EXPORT struct libinput_event_gesture *
>> +libinput_event_get_gesture_event(struct libinput_event *event)
>> +{
>> + switch (event->type) {
>> + case LIBINPUT_EVENT_NONE:
>> + abort(); /* not used as actual event type */
>> + case LIBINPUT_EVENT_DEVICE_ADDED:
>> + case LIBINPUT_EVENT_DEVICE_REMOVED:
>> + case LIBINPUT_EVENT_KEYBOARD_KEY:
>> + break;
>> + case LIBINPUT_EVENT_POINTER_MOTION:
>> + case LIBINPUT_EVENT_POINTER_MOTION_ABSOLUTE:
>> + case LIBINPUT_EVENT_POINTER_BUTTON:
>> + case LIBINPUT_EVENT_POINTER_AXIS:
>> + break;
>> + case LIBINPUT_EVENT_TOUCH_DOWN:
>> + case LIBINPUT_EVENT_TOUCH_UP:
>> + case LIBINPUT_EVENT_TOUCH_MOTION:
>> + case LIBINPUT_EVENT_TOUCH_CANCEL:
>> + case LIBINPUT_EVENT_TOUCH_FRAME:
>> + break;
>> + case LIBINPUT_EVENT_GESTURE_SWIPE_START:
>> + case LIBINPUT_EVENT_GESTURE_SWIPE_UPDATE:
>> + case LIBINPUT_EVENT_GESTURE_SWIPE_END:
>> + return (struct libinput_event_gesture *) event;
>> }
>>
>> return NULL;
>> @@ -259,6 +313,10 @@ libinput_event_get_device_notify_event(struct libinput_event *event)
>> case LIBINPUT_EVENT_TOUCH_CANCEL:
>> case LIBINPUT_EVENT_TOUCH_FRAME:
>> break;
>> + case LIBINPUT_EVENT_GESTURE_SWIPE_START:
>> + case LIBINPUT_EVENT_GESTURE_SWIPE_UPDATE:
>> + case LIBINPUT_EVENT_GESTURE_SWIPE_END:
>> + break;
>> }
>>
>> return NULL;
>> @@ -501,6 +559,56 @@ libinput_event_touch_get_y(struct libinput_event_touch *event)
>> return evdev_convert_to_mm(device->abs.absinfo_y, event->y);
>> }
>>
>> +LIBINPUT_EXPORT uint32_t
>> +libinput_event_gesture_get_time(struct libinput_event_gesture *event)
>> +{
>> + return event->time;
>> +}
>> +
>> +LIBINPUT_EXPORT int
>> +libinput_event_gesture_get_finger_count(struct libinput_event_gesture *event)
>> +{
>> + return event->finger_count;
>> +}
>> +
>> +LIBINPUT_EXPORT double
>> +libinput_event_gesture_get_dx(struct libinput_event_gesture *event)
>> +{
>> + return event->dx;
>> +}
>> +
>> +LIBINPUT_EXPORT double
>> +libinput_event_gesture_get_dy(struct libinput_event_gesture *event)
>> +{
>> + return event->dy;
>> +}
>> +
>> +LIBINPUT_EXPORT double
>> +libinput_event_gesture_get_dx_unaccelerated(
>> + struct libinput_event_gesture *event)
>> +{
>> + return event->dx_unaccel;
>> +}
>> +
>> +LIBINPUT_EXPORT double
>> +libinput_event_gesture_get_dy_unaccelerated(
>> + struct libinput_event_gesture *event)
>> +{
>> + return event->dy_unaccel;
>> +}
>> +
>> +LIBINPUT_EXPORT double
>> +libinput_event_gesture_get_absolute_x(struct libinput_event_gesture *event)
>> +{
>> + return event->abs_x;
>> +}
>> +
>> +LIBINPUT_EXPORT double
>> +libinput_event_gesture_get_absolute_y(struct libinput_event_gesture *event)
>> +{
>> + return event->abs_y;
>> +}
>> +
>> struct libinput_source *
>> libinput_add_fd(struct libinput *libinput,
>> int fd,
>> @@ -1162,6 +1270,39 @@ touch_notify_frame(struct libinput_device *device,
>> &touch_event->base);
>> }
>>
>> +void
>> +gesture_notify_swipe(struct libinput_device *device,
>> + uint64_t time,
>> + enum libinput_event_type type,
>> + int finger_count,
>> + double dx,
>> + double dy,
>> + double dx_unaccel,
>> + double dy_unaccel,
>> + double abs_x,
>> + double abs_y)
>> +{
>> + struct libinput_event_gesture *gesture_event;
>> +
>> + gesture_event = zalloc(sizeof *gesture_event);
>> + if (!gesture_event)
>> + return;
>> +
>> + *gesture_event = (struct libinput_event_gesture) {
>> + .time = time,
>> + .finger_count = finger_count,
>> + .dx = dx,
>> + .dy = dy,
>> + .dx_unaccel = dx_unaccel,
>> + .dy_unaccel = dy_unaccel,
>> + .abs_x = abs_x,
>> + .abs_y = abs_y,
>> + };
>> +
>> + post_device_event(device, time, type,
>> + &gesture_event->base);
>> +}
>> +
>> static void
>> libinput_post_event(struct libinput *libinput,
>> struct libinput_event *event)
>> @@ -1387,6 +1528,12 @@ libinput_event_touch_get_base_event(struct libinput_event_touch *event)
>> return &event->base;
>> }
>>
>> +LIBINPUT_EXPORT struct libinput_event *
>> +libinput_event_gesture_get_base_event(struct libinput_event_gesture *event)
>> +{
>> + return &event->base;
>> +}
>> +
>> LIBINPUT_EXPORT const char *
>> libinput_config_status_to_str(enum libinput_config_status status)
>> {
>> diff --git a/src/libinput.h b/src/libinput.h
>> index 7b7a2db..e1ccaf3 100644
>> --- a/src/libinput.h
>> +++ b/src/libinput.h
>> @@ -173,7 +173,11 @@ enum libinput_event_type {
>> * Signals the end of a set of touchpoints at one device sample
>> * time. This event has no coordinate information attached.
>> */
>> - LIBINPUT_EVENT_TOUCH_FRAME
>> + LIBINPUT_EVENT_TOUCH_FRAME,
>> +
>> + LIBINPUT_EVENT_GESTURE_SWIPE_START = 600,
>> + LIBINPUT_EVENT_GESTURE_SWIPE_UPDATE,
>> + LIBINPUT_EVENT_GESTURE_SWIPE_END,
>> };
>>
>> /**
>> @@ -350,6 +354,19 @@ libinput_event_get_touch_event(struct libinput_event *event);
>> /**
>> * @ingroup event
>> *
>> + * Return the gesture event that is this input event. If the event type does
>> + * not match the gesture event types, this function returns NULL.
>> + *
>> + * The inverse of this function is libinput_event_gesture_get_base_event().
>> + *
>> + * @return A gesture event, or NULL for other events
>> + */
>> +struct libinput_event_gesture *
>> +libinput_event_get_gesture_event(struct libinput_event *event);
>> +
>> +/**
>> + * @ingroup event
>> + *
>> * Return the device event that is this input event. If the event type does
>> * not match the device event types, this function returns NULL.
>> *
>> @@ -884,6 +901,186 @@ struct libinput_event *
>> libinput_event_touch_get_base_event(struct libinput_event_touch *event);
>>
>> /**
>> + * @defgroup event_gesture Gesture events
>> + *
>> + * Gesture events are generated when a gesture is recognized on a touchpad.
>> + *
>> + * Gesture sequences always start with a LIBINPUT_EVENT_GESTURE_FOO_START
>> + * event. All following gesture events will be of the
>> + * LIBINPUT_EVENT_GESTURE_FOO type until a LIBINPUT_EVENT_GESTURE_FOO_END is
>> + * generated which signals the end of the gesture.
>> + */
>> +
>> +/**
>> + * @ingroup event_gesture
>> + *
>> + * @return The event time for this event
>> + */
>> +uint32_t
>> +libinput_event_gesture_get_time(struct libinput_event_gesture *event);
>> +
>> +/**
>> + * @ingroup event_gesture
>> + *
>> + * @return The generic libinput_event of this event
>> + */
>> +struct libinput_event *
>> +libinput_event_gesture_get_base_event(struct libinput_event_gesture *event);
>> +
>> +/**
>> + * @ingroup event_gesture
>> + *
>> + * Return the number of fingers used for a gesture. This can be used e.g.
>> + * to differentiate between 3 or 4 finger swipes.
>> + *
>> + * This function can be called on all gesture events including
>> + * LIBINPUT_EVENT_GESTURE_FOO_START and the returned finger count value will
>> + * not change during a sequence.
>> + *
>> + * @return the number of fingers used for a gesture
>> + */
>> +int
>> +libinput_event_gesture_get_finger_count(struct libinput_event_gesture *event);
>> +
>> +/**
>> + * @ingroup event_gesture
>> + *
>> + * Return the delta between the last event and the current event. For gesture
>> + * events that are not of type @ref LIBINPUT_EVENT_GESTURE_SWIPE_UPDATE, this
>> + * function returns 0.
>> + *
>> + * If a device employs pointer acceleration, the delta returned by this
>> + * function is the accelerated delta.
>> + *
>> + * Relative motion deltas are normalized to represent those of a device with
>> + * 1000dpi resolution. See @ref motion_normalization for more details.
>
> If the default DPI ever changes again for whatever reason, all these
> comments need an update... Can we get away without specifying the DPI
> (since it's explained in @ref motion_normalization anyway...)
We should never change the default dpi, this is only for normalization, it is
not a dpi which we actually assign to devices with unknown dpi.
>
>> + *
>> + * @note It is an application bug to call this function for events other than
>> + * @ref LIBINPUT_EVENT_GESTURE_SWIPE_UPDATE.
>> + *
>> + * @return the relative x movement since the last event
>> + */
>> +double
>> +libinput_event_gesture_get_dx(struct libinput_event_gesture *event);
>> +
>> +/**
>> + * @ingroup event_gesture
>> + *
>> + * Return the delta between the last event and the current event. For gesture
>> + * events that are not of type @ref LIBINPUT_EVENT_GESTURE_SWIPE_UPDATE, this
>> + * function returns 0.
>> + *
>> + * If a device employs pointer acceleration, the delta returned by this
>> + * function is the accelerated delta.
>> + *
>> + * Relative motion deltas are normalized to represent those of a device with
>> + * 1000dpi resolution. See @ref motion_normalization for more details.
>> + *
>> + * @note It is an application bug to call this function for events other than
>> + * @ref LIBINPUT_EVENT_GESTURE_SWIPE_UPDATE.
>> + *
>> + * @return the relative y movement since the last event
>> + */
>> +double
>> +libinput_event_gesture_get_dy(struct libinput_event_gesture *event);
>> +
>> +/**
>> + * @ingroup event_gesture
>> + *
>> + * Return the relative delta of the unaccelerated motion vector of the
>> + * current event. For gesture events that are not of type @ref
>> + * LIBINPUT_EVENT_GESTURE_SWIPE_UPDATE, this function returns 0.
>> + *
>> + * Relative unaccelerated motion deltas are normalized to represent those of a
>> + * device with 1000dpi resolution. See @ref motion_normalization for more
>> + * details. Note that unaccelerated events are not equivalent to 'raw' events
>> + * as read from the device.
>> + *
>> + * @note It is an application bug to call this function for events other than
>> + * @ref LIBINPUT_EVENT_GESTURE_SWIPE_UPDATE.
>> + *
>> + * @return the unaccelerated relative x movement since the last event
>> + */
>> +double
>> +libinput_event_gesture_get_dx_unaccelerated(
>> + struct libinput_event_gesture *event);
>> +
>> +/**
>> + * @ingroup event_gesture
>> + *
>> + * Return the relative delta of the unaccelerated motion vector of the
>> + * current event. For gesture events that are not of type @ref
>> + * LIBINPUT_EVENT_GESTURE_SWIPE_UPDATE, this function returns 0.
>> + *
>> + * Relative unaccelerated motion deltas are normalized to represent those of a
>> + * device with 1000dpi resolution. See @ref motion_normalization for more
>> + * details. Note that unaccelerated events are not equivalent to 'raw' events
>> + * as read from the device.
>> + *
>> + * @note It is an application bug to call this function for events other than
>> + * @ref LIBINPUT_EVENT_GESTURE_SWIPE_UPDATE.
>> + *
>> + * @return the unaccelerated relative y movement since the last event
>> + */
>> +double
>> +libinput_event_gesture_get_dy_unaccelerated(
>> + struct libinput_event_gesture *event);
>> +
>> +/**
>> + * @ingroup event_gesture
>> + *
>> + * Return the absolute x coordinate of the current center position of a
>> + * swipe gesture. This can be used e.g. to determine if a swipe is starting
>> + * close to a touchpad edge, or to synchronize an animation with how many
>> + * percent of the width of a touchpad a swipe gesture has traveled.
>> + *
>> + * The returned value is in the range of 0.0 to 1.0 with 0.0 indicating that
>> + * the center is on the right edge of the touchpad, and 1.0 indicating that it
>> + * is on the left edge.
>
> 0.0 on the right, 1.0 on the left? Is there a reason it's the opposite
> of screen co-ordinates?
Er, no actually I got this wrong it is intended to be the otherway around,
as that also matches the coordinate space of (most) touchpads. My bad, this
will change actually to give a location in mm, with a helper function to
convert it to a % scale (or some such) for people who want that.
>
>> + *
>> + * For gesture events that are not of type
>> + * @ref LIBINPUT_EVENT_GESTURE_SWIPE_START,
>> + * @ref LIBINPUT_EVENT_GESTURE_SWIPE_UPDATE or
>> + * @ref LIBINPUT_EVENT_GESTURE_SWIPE_END this function returns 0.
>> + *
>> + * @note It is an application bug to call this function for events other than
>> + * @ref LIBINPUT_EVENT_GESTURE_SWIPE_START,
>> + * @ref LIBINPUT_EVENT_GESTURE_SWIPE_UPDATE or
>> + * @ref LIBINPUT_EVENT_GESTURE_SWIPE_END.
>> + *
>> + * @return the current absolute x coordinate
>> + */
>> +double
>> +libinput_event_gesture_get_absolute_x(struct libinput_event_gesture *event);
>> +
>> +/**
>> + * @ingroup event_gesture
>> + *
>> + * Return the absolute y coordinate of the current center position of a
>> + * swipe gesture. This can be used e.g. to determine if a swipe is starting
>> + * close to a touchpad edge, or to synchronize an animation with how many
>> + * percent of the height of a touchpad a swipe gesture has traveled.
>> + *
>> + * The returned value is in the range of 0.0 to 1.0 with 0.0 indicating that
>> + * the center is on the top edge of the touchpad, and 1.0 indicating that it
>> + * is on the bottom edge.
>> + *
>> + * For gesture events that are not of type
>> + * @ref LIBINPUT_EVENT_GESTURE_SWIPE_START,
>> + * @ref LIBINPUT_EVENT_GESTURE_SWIPE_UPDATE or
>> + * @ref LIBINPUT_EVENT_GESTURE_SWIPE_END this function returns 0.
>> + *
>> + * @note It is an application bug to call this function for events other than
>> + * @ref LIBINPUT_EVENT_GESTURE_SWIPE_START,
>> + * @ref LIBINPUT_EVENT_GESTURE_SWIPE_UPDATE or
>> + * @ref LIBINPUT_EVENT_GESTURE_SWIPE_END.
>> + *
>> + * @return the current absolute y coordinate
>> + */
>> +double
>> +libinput_event_gesture_get_absolute_y(struct libinput_event_gesture *event);
>> +
>> +/**
>> * @defgroup base Initialization and manipulation of libinput contexts
>> */
>>
>>
>
Regards,
Hans
More information about the wayland-devel
mailing list